Functional Programming: Map, Filter, and Reduce

Functional programming is a programming paradigm where we treat computation as the evaluation of mathematical functions, avoiding changing-state and mutable data.

In data analytics and data science, you frequently need to process large sequences of data (like columns in a spreadsheet or log files). Python provides three extremely powerful built-in functions to perform these sequence operations efficiently: map(), filter(), and reduce().

1. The map() Function

The map() function applies a specified function to each item of an iterable (like a list) and returns a map object (an iterator).

Example: Scaling Feature Values (Data Preprocessing)

In machine learning, we often scale values (like house prices or user ratings) to a specific range (e.g., Min-Max scaling or simple division by a scaling factor) so the algorithm performs better.

# List of house prices in a neighborhood (in raw USD)
house_prices = [250000, 320000, 180000, 450000, 600000]

# Function to scale price to "Thousands of USD"
def scale_to_thousands(price):
    return price / 1000.0

# Applying map to scale the entire dataset
scaled_prices = list(map(scale_to_thousands, house_prices))

print("Raw Prices:   ", house_prices)
print("Scaled Prices (k$):", scaled_prices)

Output:

Raw Prices:    [250000, 320000, 180000, 450000, 600000]
Scaled Prices (k$): [250.0, 320.0, 180.0, 450.0, 600.0]

2. The filter() Function

The filter() function constructs an iterator from elements of an iterable for which a function returns True.

Example: Fraud Detection (Transaction Filtering)

As a financial data analyst, you want to identify all transactions that exceed a high-risk threshold (e.g., transactions greater than $5,000) for closer audit.

# Financial transaction amounts
transactions = [120.50, 6200.00, 45.00, 8900.25, 1200.00, 5005.10]

# Rule function for high-risk threshold
def is_high_risk(amount):
    return amount > 5000.00

# Filtering out low-risk transactions
flagged_transactions = list(filter(is_high_risk, transactions))

print("All Transactions:    ", transactions)
print("Flagged (High Risk):", flagged_transactions)

Output:

All Transactions:     [120.5, 6200.0, 45.0, 8900.25, 1200.0, 5005.1]
Flagged (High Risk): [6200.0, 8900.25, 5005.1]

3. The reduce() Function

Unlike map() and filter(), reduce() is not a global built-in function; it must be imported from the functools module.

reduce() repeatedly applies a binary function (a function taking two arguments) to the elements of a sequence, from left to right, reducing the sequence to a single cumulative value.

Example A: Custom Reducer

When calculating complex mathematical compositions—like compounding varying annual investment returns on an initial capital—Python has no built-in function to help. You must write a custom reducer function.

from functools import reduce

# Varying annual returns over 4 years: +5%, +12%, -3%, +8%
annual_returns = [0.05, 0.12, -0.03, 0.08]

# Custom compounding reducer function
# balance represents the accumulated balance, rate is the next year's return
def compound_growth(balance, rate):
    return balance * (1 + rate)

# Initial capital investment is $10,000
initial_investment = 10000

# Reducing the returns down to a final capital amount
final_balance = reduce(compound_growth, annual_returns, initial_investment)

print("Annual Return Rates:  ", annual_returns)
print(f"Final Investment Value: ${final_balance:,.2f}")

Output:

Annual Return Rates:   [0.05, 0.12, -0.03, 0.08]
Final Investment Value: $12,319.78

Example B: Reducer using a Built-in Function

If you want to perform a standard binary operation across a sequence that doesn't have a single aggregate function (like multiplication, where Python has sum() for addition but no standard global multiplication counterpart), you can pair reduce() with a function from the built-in operator module.

from functools import reduce
from operator import mul

# List of scaling dimensions / factors
multipliers = [2, 3, 1.5, 5]

# Using the built-in mul (multiplication) operator to calculate the product
total_product = reduce(mul, multipliers)

print("Multipliers:  ", multipliers)
print("Total Product:", total_product)

Output:

Multipliers:   [2, 3, 1.5, 5]
Total Product: 45.0