Surendra Sharma

Surendra Sharma

Search This Blog

Thursday, October 26, 2023

Whitepaper : Sitecore Pipeline

Pipeline in general
Pipeline in general

In programming, a pipeline is a concept used to describe a sequence of data processing elements or functions that are connected in a chain, where the output of one element serves as the input for the next element. This technique is commonly used to organize and streamline the flow of data or operations in various computational processes.

Pipeline Flow
Pipeline Flow


In a pipeline, data flows through different stages or processing units, with each stage performing a specific operation on the data. Each stage typically focuses on a particular task, and the combination of all stages allows for complex data processing or transformations to be achieved efficiently.

The pipeline pattern is widely used in various areas of software development, such as data processing, data analysis, and even web application development.

 

Examples of Pipelines

 

Data Processing Pipeline: In data processing, a pipeline might involve several stages for transforming and analyzing data. For example, in a data analysis pipeline, data could flow through stages like data ingestion, cleaning, feature extraction, machine learning model training, and finally, result evaluation.

Image Processing Pipeline: In image processing, a pipeline could be used to process images through various stages, such as image resizing, noise reduction, color correction, edge detection, and image enhancement.

Web Development Pipeline: In web development, a pipeline can be used for tasks like asset compilation and optimization. For instance, a pipeline could involve concatenating and minifying CSS and JavaScript files, optimizing images, and caching assets for faster loading times.

Continuous Integration/Continuous Deployment (CI/CD) Pipeline: In software development, a CI/CD pipeline automates the process of building, testing, and deploying code changes. This pipeline might include stages like code compilation, unit testing, integration testing, and finally, deployment to production servers.

Natural Language Processing (NLP) Pipeline: In NLP, a pipeline could be employed to process text data through various stages, such as tokenization, part-of-speech tagging, named entity recognition, sentiment analysis, and text summarization.

Audio Processing Pipeline: In audio processing, a pipeline might involve stages for audio file decoding, noise reduction, audio feature extraction, speech-to-text conversion, and text-to-speech synthesis.

Video Editing Pipeline: In video editing, a pipeline could include stages like video clipping, adding special effects, transitions, audio synchronization, and video rendering.

These examples demonstrate how pipelines can be used to break down complex tasks into manageable steps, where each step in the pipeline performs a specific operation, and the output of one step serves as the input for the next step. This modular approach to programming allows for easier development, testing, and maintenance of software systems.

 

Introduction of Sitecore Pipeline

 

Sitecore Pipeline is a fundamental concept of the Sitecore CMS architecture. It's a way of defining a sequence of steps that execute in a particular order to handle specific events or processes within the Sitecore request processing lifecycle. This approach allows for highly extensible and modular development of Sitecore solutions.

In this article, I will provide an in-depth look at Sitecore Pipeline with practical examples of how to leverage this architecture to extend and customize Sitecore functionality.


Understanding Sitecore Pipeline

Before diving into Sitecore Pipeline, it's essential to understand the Sitecore request processing lifecycle. Sitecore request processing lifecycle has various stages, and each stage is a specific event that occurs within Sitecore.

When an HTTP request arrives at a Sitecore website, the request is processed through various stages before rendering a response. Each stage consists of one or more pipelines, and each pipeline consists of one or more processors.

A processor is a C# method that executes a specific operation or handles a particular event within the request processing lifecycle. The processor can modify the request, perform additional operations or even prevent the request from proceeding to the next stage of the lifecycle.


Sitecore Pipeline Architecture

Sitecore Pipeline architecture is based on the idea of a sequence of processors that execute in a specific order. The sequence of processors is grouped into a pipeline. Each pipeline executes at a specific stage in the Sitecore request processing lifecycle.

Pipeline and Processors
Pipeline and Processors


Few types of pipelines in Sitecore are:

InitializePipeline - Executes when an Sitecore instance start.

HttpPipeline - Executes when an HTTP request is made to the Sitecore website.

PublishPipeline - Executes when the Sitecore items are being publish.

Each pipeline consists of a sequence of processors that execute in a specific order. These processors can be Sitecore-built-in processors, custom processors, or a combination of both.


Creating Custom Processors

Creating custom processors is a straightforward process. You create a C# method that performs the operation you want and then add it to the pipeline.

The first step is to define the pipeline where the processor will execute. You can add a processor to an existing pipeline or create a new pipeline.

Once you have identified the pipeline, you will define the processor. In this step, you will create a C# method that performs the operation you want.

After defining the processor, you will register it with the pipeline. You can register the processor at the beginning or the end of the pipeline or somewhere in between.


Best Practices for Sitecore Pipeline

Here are some best practices to keep in mind when working with Sitecore pipelines:

Plan Your Pipeline Carefully: Plan your pipeline carefully to ensure that processors execute in the correct order and that they do not conflict with each other.

Keep Your Pipeline Simple: Keep your pipeline simple by breaking it down into small, reusable processors.

Use Sitecore-Built-in Processors: Whenever possible.


Pipeline Examples

Now that we have a good understanding of how pipelines work, let's take a look at some examples of how we can use pipelines to perform specific tasks.

 

1.      Http Request Processor for Page not found

 

This example shows that how to create custom Page not found for Sitecore website.

 

using Sitecore.Data.Items;

using Sitecore.Pipelines.HttpRequest;

 

namespace AlexaSearch.Pipelines

{

    public class HttpRequestProcessor404 : HttpRequestProcessor

    {

        public override void Process(HttpRequestArgs args)

        {

            if (Sitecore.Context.Item != null || Sitecore.Context.Site == null || Sitecore.Context.Database == null

                || Sitecore.Context.Database.Name == "core"

                || args.RequestUrl.AbsoluteUri.ToLower().Contains("/sitecore/api/layout/render/jss")

                || args.RequestUrl.AbsoluteUri.ToLower().Contains("/sitecore/api/jss/import")

                || args.RequestUrl.AbsoluteUri.ToLower().Contains("/api/sitecore/"))

            {

                return;

            }

 

            var alexa404PageId = Sitecore.Configuration.Settings.GetSetting("Alexa404");

            var pageNotFound = Sitecore.Context.Database.GetItem(alexa404PageId);

            if (pageNotFound == null)

                return;

            args.ProcessorItem = pageNotFound;

            Sitecore.Context.Item = pageNotFound;

            args.HttpContext.Response.StatusCode = 404;

        }

    }

}

 

Web.config entry

<pipelines>

                                       

             <httpRequestBegin>

                           <processor type="AlexaSearch.Pipelines.HttpRequestProcessor404,AlexaSearch"

                          patch:after="processor[@type='Sitecore.Pipelines.HttpRequest.ItemResolver, Sitecore.Kernel']"/>

            

             </httpRequestBegin>

 

 

2.      Item Processed Processor for Publish Items

 

This example shows that how to create processor to get publish items and logging their activities.

 

using Sitecore.Data.Items;

using Sitecore.Diagnostics;

using Sitecore.Publishing.Pipelines.Publish;

using System;

using System.Collections.Concurrent;

using System.Collections.Generic;

using System.Linq;

using System.Web.UI;

 

namespace AlexaSearch.Pipelines

{

    public class ItemProcessedProcessor : PublishProcessor

    {

        public override void Process(PublishContext context)

        {

            if (!context.Aborted)

            {

                var publishItems =  context.ProcessedPublishingCandidates.Keys;

                foreach(var tempItem in publishItems)

                {

                    var currentItem = context.PublishOptions.TargetDatabase.GetItem(tempItem.ItemId);

 

                    if(currentItem != null)

                        Sitecore.Diagnostics.Log.Info("My Publish item is " + currentItem.Name, this);

                }

            }

        }

    }

}

 

Web.config entry

<pipelines>

                                       

             <publish>

                           <processor patch:after="*[@type='Sitecore.Publishing.Pipelines.Publish.ProcessQueue, Sitecore.Kernel']" type="AlexaSearch.Pipelines.ItemProcessedProcessor, AlexaSearch" />

             </publish>

 

3.      Log Request Begin processor

 

This example shows that how to log HttpRequestBegin pipeline activities.

 

using Sitecore.Pipelines.HttpRequest;

using System;

using System.Collections.Generic;

using System.Linq;

using System.Web;

 

namespace AlexaSearch.Pipelines

{

    public class LogRequestBegin

    {

        public void Process(HttpRequestArgs args)

        {

            Sitecore.Diagnostics.Log.Error(DateTime.Now + " Hello HttpRequestBegin pipeline executed ", this);

        }

    }

}

 

Web.config entry

<pipelines>

                                       

             <httpRequestBegin>

                           <processor type="AlexaSearch.Pipelines.LogRequestBegin, AlexaSearch" /> 

             </httpRequestBegin>

 

4.      Custom Field Processor

 

This example shows that how to capture each fields in processor. It will check for rich text field and append extra text at the end of rich text field content.

 

using Sitecore.Pipelines.RenderField;

using System;

using System.Collections.Generic;

using System.Linq;

using System.Web;

 

namespace AlexaSearch.Pipelines

{

    public class MyCustomFieldProcessor

    {

        public void Process(RenderFieldArgs args)

        {

            if (!(args.FieldTypeKey == "rich text"))

                return;

            args.After = "<div><h3>Copyright Alexa</h3></div>";

        }

    }

}

 

Web.config entry

<pipelines>

                                       

             <renderField>

                           <processor  patch:after="processor[@type='Sitecore.Pipelines.RenderField.GetTextFieldValue, Sitecore.Kernel']"

                                                     type="AlexaSearch.Pipelines.MyCustomFieldProcessor, AlexaSearch" />

             </renderField>

 

5.      Set Default Alternate Text in media item

 

This example shows that how to Set Default Alternate Text in media item.

 

using Sitecore.Diagnostics;

using Sitecore.Pipelines.GetMediaCreatorOptions;

 

namespace AlexaSearch.Pipelines

{

    public class SetDefaultAlternateTextIfNeed

    {

        public void Process(GetMediaCreatorOptionsArgs args)

        {

            Assert.ArgumentNotNull(args, "args");

            if (!string.IsNullOrWhiteSpace(args.Options.AlternateText))

            {

                return;

            }

 

            args.Options.AlternateText = GetAlternateText(args);

        }

 

        protected virtual string GetAlternateText(GetMediaCreatorOptionsArgs args)

        {

            Assert.ArgumentNotNull(args, "args");

            if (string.IsNullOrWhiteSpace(args.Options.Destination) || args.Options.Destination.IndexOf("/") < 0)

            {

                return string.Empty;

            }

 

            int startofNameIndex = args.Options.Destination.LastIndexOf("/") + 1;

            return args.Options.Destination.Substring(startofNameIndex);

        }

    }

}

 

Web.config entry

<pipelines>

                                       

             <getMediaCreatorOptions>

                           <processor type="AlexaSearch.Pipelines.SetDefaultAlternateTextIfNeed, AlexaSearch"/>

             </getMediaCreatorOptions>

 

What is CorePipeline.Run()


CorePipeline.Run() is used in Sitecore when you want to execute a pipeline programmatically. Pipelines in Sitecore are used to execute a series of processors in a specific order. They are defined in configuration files and are typically triggered by specific events or user actions.

Here are some situations where you might want to use CorePipeline.Run():

Implementing custom business logic: If you have custom business logic that needs to be executed as part of a Sitecore pipeline, you can create a new pipeline and use CorePipeline.Run() to execute it.

Modifying the behavior of existing Sitecore pipelines: If you need to modify the behavior of an existing Sitecore pipeline, you can add your own processor to the pipeline and use CorePipeline.Run() to execute it.

Triggering pipelines programmatically: If you need to trigger a pipeline programmatically, for example in response to a user action, you can use CorePipeline.Run() to do so.

Integrating with third-party systems: If you need to integrate Sitecore with a third-party system that requires custom processing, you can create a new pipeline and use CorePipeline.Run() to execute it.

Example

 

using Sitecore.Pipelines;

using System;

 

namespace AlexaSearch.Pipelines

{

    public class MyCustomPipelineArgs : PipelineArgs

    {

        public string CustomValue { get; set; }

    }

 

    public class MyCustomPipelineProcessor

    {

        public void Process(MyCustomPipelineArgs args)

        {

            Sitecore.Diagnostics.Log.Error(DateTime.Now + " My Custom Pipeline Processor " + args.CustomValue, this);

        }

    }

}

 

Call this pipeline from some Views as

@{

            var args = new AlexaSearch.Pipelines.MyCustomPipelineArgs { CustomValue = "Hello, World!" };

            Sitecore.Pipelines.CorePipeline.Run("myCustomPipeline", args);

}

                          

                           <div>

                                        <p>=====================================================</p><br>

                                        This is my Footer Component<br>

                                        <p>=====================================================</p><br>

                           </div>  

 

Web.config entry

<pipelines>

                                       

             <myCustomPipeline>

                           <processor type="AlexaSearch.Pipelines.MyCustomPipelineProcessor, AlexaSearch" />

             </myCustomPipeline>

 

Monitor Sitecore pipelines

 

There are a few ways to monitor Sitecore pipelines:

Log files: Sitecore writes detailed log files that can be used to monitor pipelines. By default, Sitecore logs are located in the App_Data/Logs folder. You can configure the logging level in the Sitecore configuration file to get more detailed information.

Debug mode: You can also monitor pipelines in debug mode by setting breakpoints in your custom pipeline processors and stepping through the code. This can be useful for troubleshooting issues or testing new pipeline functionality.

Pipeline Profiler: It is a Sitecore tool that helps developers to monitor the performance of pipelines in real-time

 

Pipeline Profiler

Pipeline Profiler
Pipeline Profiler

Pipeline Profiler is a Sitecore tool that helps developers to monitor the performance of pipelines in real-time. It is a visual representation of the Sitecore pipeline events that occur during an interaction with a website, such as page rendering or a form submission.

The Pipeline Profiler can be accessed it from /sitecore/admin/pipelines.aspx. It displays the execution time for each pipeline and the time taken by each processor within that pipeline. This information can be used to identify performance bottlenecks in the pipeline and to optimize pipeline execution.

 

Advantages of Sitecore Pipeline:

 

Flexibility: Sitecore pipelines provide a flexible way to add, remove, or modify functionality at different stages of a process. This allows for customizations without impacting the existing codebase.

Reusability: Since pipelines are modular, they can be reused in different contexts, making development faster and more efficient.

Maintainability: Separating code into smaller, more manageable pieces makes it easier to maintain and debug.

Extensibility: Pipelines can be extended with new processors to meet changing business requirements.


Disadvantages of Sitecore Pipeline:

Overhead: Pipelines can add some overhead to processing times, especially if there are many processors involved.

Complexity: Understanding the Sitecore pipeline model and how to work with it can be complex and requires some experience.

Performance: While pipelines can be used to optimize performance, if not implemented carefully, they can have a negative impact on performance.

Debugging: Debugging pipeline issues can be challenging because of the number of processors involved and the complex interactions between them.

 

When not to use Sitecore pipelines?

While Sitecore pipelines offer a lot of benefits, there are some situations where they might not be the best fit. Here are some cases where using pipelines may not be ideal:

Simple, straightforward tasks: If a task is simple and straightforward, it may not require the use of a pipeline. For example, if you only need to update a single item, you could do that directly without creating a pipeline.

Performance-critical tasks: While Sitecore pipelines are very flexible and powerful, they do have some overhead associated with them. If you have a task that needs to be performed as quickly as possible, using a pipeline may add too much overhead and slow things down.

Tight coupling between components: If components within your application are tightly coupled, it may not make sense to use a pipeline. Pipelines are designed to loosely couple components, so if there is already a tight coupling between the components, adding a pipeline may just add unnecessary complexity.

Inflexible requirements: If the requirements for a task are very rigid and not likely to change, it may not make sense to use a pipeline. Pipelines are designed to be very flexible and adaptable, so if the requirements are unlikely to change, you may not need that level of flexibility.


Conclusion

In this article, we have explored the Sitecore pipeline framework and how it can be used to perform various tasks in Sitecore. I have covered the basics of how pipelines work, how to configure them in Sitecore, and some examples of how they can be used.

By understanding the Sitecore pipeline framework and how to use it effectively, developers can create more powerful and flexible solutions in Sitecore. With this knowledge, you will be able to leverage the pipeline.

Thursday, September 14, 2023

Authentication failed because the remote party has closed the transport stream

Sitecore Content Hub is a powerful tool for managing digital assets and content. When it comes to integrating Content Hub with your Sitecore instance, you may encounter some weird challenges. One of the issues that users may face is the "Authentication failed because the remote party has closed the transport stream" error. In this article, we will explore the causes of this error and how to resolve it.

 

The Scenario

Imagine you have a Sitecore instance running on Azure PaaS, and you've installed the Content Hub connector. Everything seems to be working smoothly – you can retrieve images and links from the Digital Asset Management (DAM) system and display them in Image and Rich Text fields. However, trouble arises when you attempt to use the "Insert DAM link" feature within the General Link field. Instead of successfully inserting a link, you encounter the perplexing "Authentication failed" error.

 

General link field error for Sitecore Content Hub
General link field error for Sitecore Content Hub

Initial Troubleshooting

Naturally, our first instinct is to check our credentials. We've logged into Content Hub in another browser tab, and there don't seem to be any issues with authentication for Image and Rich Text fields. The problem only arises when interacting with the General Link field as

 

Checking internal code

Digging deeper into the issue, we examine the code responsible for communicating with the Content Hub API. This line of code is executed exclusively when we use the "Insert DAM link" command within the General Link field which basically retrieves additional properties.

value = GetAdditionalPropertyValue(chKey, DAMHelper.GetEntity(long.Parse(result[Constants.SelectedAssetId])));

 

Suspecting TLS Issue

At this point, we might suspect a Transport Layer Security (TLS) issue, as TLS configuration can sometimes lead to connectivity problems. However, we verify that our development environment is configured to use TLS 1.2, eliminating TLS as the root cause.

 

Azure VNET to the Rescue

With conventional troubleshooting avenues exhausted, we turn our attention to our Azure Virtual Network (VNET) configuration. After reviewing our VNET traffic rules, we discover the key to resolving the issue. We open up the traffic for the Sitecore Content Hub URL at the Azure VNET level.

 

The Solution

To our relief, this Azure VNET setting proves to be the missing piece of the puzzle. Once we've made the necessary adjustments, the "Authentication failed" error becomes a thing of the past. The General Link field now functions seamlessly, and we can insert DAM links without encountering any connectivity problems.

 

Finally

Troubleshooting errors like "Authentication failed because the remote party has closed the transport stream" can be a complex process. It often requires a systematic approach to identify the root cause and find a solution. In this case, a careful examination of the code, consideration of TLS configurations, and ultimately, the adjustment of Azure VNET settings, resolved the issue.

By sharing this experience, we hope to assist others who may encounter similar challenges when using Sitecore Content Hub connectors. Remember that troubleshooting can be a learning opportunity, and persistence often leads to a successful resolution.

Monday, September 4, 2023

Troubleshooting Sitecore Content Hub Connector Authentication Issue with General Link Fields

Sitecore Content Hub Connector is a powerful tool that enables seamless integration between Sitecore and Content Hub. This integration allows content authors to easily access and manage digital assets from Content Hub within Sitecore. However, like any integration, it can encounter occasional issues. In this blog post, we'll discuss a specific problem some users have faced: an authentication error when working with General Link fields in Sitecore.

 

The Scenario:

You've set up Sitecore 10.3 on Azure PaaS and successfully integrated it with Sitecore Content Hub using PowerShell installation script. Everything seems to be working fine like RichText and Image field able to get images from Content Hub successfully. But while using the "Insert DAM Link" option within a General Link field, you encounter an error message that says, "Unable to authorize with the specified credentials."

 

authentication error
Authentication error

Troubleshooting the Issue:

Let's walk through the steps to diagnose and resolve it.

Review Connection String:

The actual Content Hub user password is "123456789$ASDfghjkl", but when I checked connection string, I notice that the password for the Content Hub user is truncated, like "123456789$". This "$" sign in the password can be the root cause of the problem.

Password Correction:

To resolve this issue, simply correct the password in your connectionstring.config. Change it to "123456789$ASDfghjkl " or whatever your actual password is without the "$" sign.

Root Cause:

PowerShell script have a problem with "$" sign. PowerShell script can truncate your password and update your ConnectionStrings.config file with wrong password. So need to update connectionstring.config file manually with correct and full password.

Testing:

After making this correction, test the "Insert DAM Link" functionality in the General Link field again. You should no longer encounter the authentication error.

Lessons Learned:

Never keep $ sign in password while installing and running Powershell scripts for installing Sitecore Content Hub Connector. 

Thursday, August 31, 2023

Connection to Your Rendering Host Failed with a Body Exceeded 2MB Limit

In the ever-evolving landscape of digital experiences, Sitecore JSS (JavaScript Services) combined with Next.js has emerged as a powerful duo for building headless websites. This approach provides flexibility, scalability, and performance. However, as with any technology, challenges may arise. 

In this blog post, we'll delve into one such challenge: the "Connection to Your Rendering Host Failed with a Body Exceeded 2MB Limit" error and how to tackle it effectively.

Understanding the Error

We have deployed our Next.js app on Azure Web App and we started to get the “Connection to your rendering host failed with a Body exceeded 2mb limit error. Ensure the POST endpoint at URL https://mynextjs-sitecore-test.azurewebapp.net/api/editing/render has been enabled” error. Its typically occurs when a request made to the Sitecore rendering host exceeds the predefined request size limit. This limit is set at 2MB by default to prevent excessively large requests from overwhelming the server and causing performance issues.

When building a Sitecore JSS-based headless website with Next.js, it's common to transfer data between the front end (Next.js) and the back end (Sitecore) using API calls. These API calls may involve sending large payloads, especially when dealing with extensive content or complex data structures.

Solutions

Once you've diagnosed the issue, it's time to implement a solution. We try to solve it in lots of 
different way, but ultimately the solution that work for us is

• Open “\src\pages\api\editing\render.ts” file and change size from 2 MB to 50 MB as

export const config = {
api: {
bodyParser: {
sizeLimit: '50mb',
},
responseLimit: false,
},
};

• Open “\src\pages\api\editing\data\[key].ts” file and change size from 2 MB to 50 MB as

export const config = {
api: {
bodyParser: {
sizeLimit: '2mb',
},
responseLimit: false,
},
};

• Build and deploy your application, it should work now.

I hope this trick may solve your problem as well. Happy coding!