Master C# Programming From Scratch

Parallel Processing

Parallel processing is a programming technique in which multiple operations are executed at the same time to improve performance and reduce the total execution time.

C# provides the Task Parallel Library (TPL), Parallel class, and other APIs that make it easier to execute independent operations concurrently.

What is Parallel Processing?

Parallel processing means dividing a large operation into smaller independent operations and allowing multiple operations to execute concurrently.

For example, suppose an application needs to process thousands of independent records. Instead of processing every record one after another, the work can be divided into smaller parts and processed concurrently.

Sequential Processing vs Parallel Processing

In sequential processing, operations are performed one after another. In parallel processing, independent operations may execute concurrently.

Parallel Class

The Parallel class is available in System.Threading.Tasks and provides simple methods for running iterations or actions concurrently.

Commonly used methods include:

• Parallel.For()
• Parallel.ForEach()
• Parallel.Invoke()

Parallel.For()

Parallel.For() executes iterations of a loop concurrently whenever the runtime determines that parallel execution is beneficial.

using System;

using System.Threading.Tasks;

class Program
{
    static void Main()
    {
        Parallel.For(1, 6, i =>
        {
            Console.WriteLine("Processing: " + i);
        });
    }
}

Possible Output

Processing: 3
Processing: 1
Processing: 4
Processing: 2
Processing: 5

How Parallel.For() Works

Instead of waiting for every iteration to finish before starting the next one, the runtime can divide the work among available worker threads.

Parallel.ForEach()

Parallel.ForEach() is useful when working with a collection and each item can be processed independently.

using System;
 
using System.Collections.Generic;
using System.Threading.Tasks;

class Program
{
    static void Main()
    {
        List<string> students = new List<string>
        {
            "Amit",
            "Priya",
            "Rahul",
            "Sneha",
            "Neha"
        };

        Parallel.ForEach(students, student =>
        {
            Console.WriteLine("Processing: " + student);
        });
    }
}

Possible Output

Processing: Rahul
Processing: Amit
Processing: Sneha
Processing: Priya
Processing: Neha

Parallel.Invoke()

Parallel.Invoke() executes multiple independent actions concurrently.

using System;
 
using System.Threading.Tasks;

class Program
{
    static void Main()
    {
        Parallel.Invoke(
            () => PrintMessage("Task A"),
            () => PrintMessage("Task B"),
            () => PrintMessage("Task C")
        );
    }

    static void PrintMessage(string message)
    {
        Console.WriteLine(message);
    }
}

Possible Output

Task B
Task A
Task C

The three actions are independent, so their execution order can vary.

Controlling Parallelism

Sometimes an application should limit how many operations execute concurrently. This can be useful when the workload is large or when system resources need to be controlled.

ParallelOptions can be used to configure parallel execution.

using System;
 
using System.Threading.Tasks;

class Program
{
    static void Main()
    {
        ParallelOptions options = new ParallelOptions
        {
            MaxDegreeOfParallelism = 2
        };

        Parallel.For(1, 7, options, i =>
        {
            Console.WriteLine("Processing item: " + i);
        });
    }
}

Explanation

In this example, the application allows a maximum of two parallel operations at a time.

Parallel Processing and Shared Data

When multiple operations work concurrently, shared mutable data can become difficult to manage safely.

For example, multiple parallel operations should not blindly update the same normal collection without considering thread safety.

using System;
 
using System.Collections.Concurrent;
using System.Threading.Tasks;

class Program
{
    static void Main()
    {
        ConcurrentBag<int> numbers = new ConcurrentBag<int>();

        Parallel.For(1, 6, i =>
        {
            numbers.Add(i);
        });

        foreach (int number in numbers)
        {
            Console.WriteLine(number);
        }
    }
}

ConcurrentBag<T> is designed for scenarios where multiple threads may add or remove items concurrently.

Parallel.ForEach() with Filtering

Parallel processing can also be combined with conditions when only certain items need to be processed.

using System;
 
using System.Collections.Generic;
using System.Threading.Tasks;

class Program
{
    static void Main()
    {
        List<int> numbers = new List<int>
        {
            10, 15, 20, 25, 30
        };

        Parallel.ForEach(numbers, number =>
        {
            if (number % 10 == 0)
            {
                Console.WriteLine("Valid: " + number);
            }
        });
    }
}

Output

Valid: 10
Valid: 20
Valid: 30

Example: Processing Student Scores

Consider an application that needs to process scores for many students. Each student's calculation is independent, so the calculations can be performed concurrently.

using System;
 
using System.Collections.Generic;
using System.Threading.Tasks;

class Program
{
    static void Main()
    {
        List<int> scores = new List<int>
        {
            75, 82, 91, 68, 88
        };

        Parallel.ForEach(scores, score =>
        {
            string result = score >= 70 ? "Pass" : "Fail";

            Console.WriteLine(
                "Score: " + score + " - " + result
            );
        });
    }
}

Possible Output

Score: 91 - Pass
Score: 75 - Pass
Score: 68 - Fail
Score: 88 - Pass
Score: 82 - Pass

Parallel Processing vs Multithreading

Parallel Processing vs Asynchronous Programming

Parallel processing and asynchronous programming solve different problems, although they can sometimes be used together.

When Should You Use Parallel Processing?

• When operations are independent.
• When the workload is large enough to benefit from concurrency.
• When the work is CPU-intensive.
• When multiple CPU cores can be used effectively.
• When the overhead of parallel execution is justified.

When Should You Avoid Parallel Processing?

• When operations depend heavily on each other.
• When the workload is extremely small.
• When shared mutable data is difficult to protect.
• When ordering is critical.
• When parallel execution creates unnecessary resource contention.

Exception Handling in Parallel Operations

Parallel operations can encounter exceptions. These exceptions need to be handled appropriately by the application.

using System;
using System.Threading.Tasks;

class Program
{
    static void Main()
    {
        try
        {
            Parallel.For(1, 6, i =>
            {
                if (i == 3)
                {
                    throw new InvalidOperationException(
                        "Invalid operation."
                    );
                }

                Console.WriteLine("Processing: " + i);
            });
        }
        catch (AggregateException ex)
        {
            foreach (Exception error in ex.InnerExceptions)
            {
                Console.WriteLine(error.Message);
            }
        }
    }
}

Parallel operations may report failures through AggregateException, which can contain one or more exceptions produced during the parallel operation.

Applications 🌍🤓

• Large-scale data processing.
• Image and video processing.
• Report generation.
• Scientific calculations.
• Financial calculations.
• Batch processing.
• Data transformation.
• Machine learning preprocessing.

Common Mistakes

1. Assuming that parallel execution always makes code faster.
2. Assuming that parallel loop iterations execute in order.
3. Updating shared data without considering thread safety.
4. Creating too much parallel work for a small operation.
5. Ignoring CPU and memory consumption.
6. Using parallel processing for operations that depend on previous results.

Best Practices

• Use parallel processing for independent workloads.
• Keep parallel operations reasonably small and focused.
• Avoid unnecessary shared mutable state.
• Use thread-safe collections when required.
• Do not depend on execution order.
• Control the degree of parallelism when necessary.
• Measure performance instead of assuming improvement.

Interview Questions

Practice Programs

1. Use Parallel.For() to print numbers from 1 to 20.
2. Use Parallel.ForEach() to process a list of student names.
3. Use Parallel.Invoke() to execute three independent methods.
4. Create a program that processes 100 numbers in parallel.
5. Use ParallelOptions to limit parallelism to two operations.
6. Use ConcurrentBag<T> with a parallel loop.
7. Compare sequential and parallel processing for a large calculation.