When you build a serverless data pipeline, Amazon API Gateway is the front door. It receives incoming HTTP requests and needs to know where to send them. In most serverless architectures, that destination is an AWS Lambda function. The crucial decision you'll make is how API Gateway communicates with Lambda. This choice defines the structure of your data, the responsibilities of your function, and the overall flexibility of your API.

There are two primary integration patterns: Lambda Proxy and Lambda Custom Integration (often called Non-Proxy). Each serves a different purpose and comes with its own set of trade-offs.

The Lambda Proxy integration is the simplest and most common approach. It acts like a transparent pass-through. API Gateway bundles the entire incoming HTTP request—headers, query string parameters, path, and body—into a single JSON object and sends it directly to your Lambda function.

This gives your function maximum context and control. You have the full, unaltered request right there in the event object. The trade-off? Your function is now responsible for interpreting that raw request and crafting a precise HTTP response. This adheres to the Single Responsibility Principle (SRP), as your function is solely responsible for handling the HTTP logic.

The event object your Lambda receives has a well-defined structure. Here’s a simplified example of what it looks like:

Notice that the body is a JSON string, so you'll need to parse it within your function. Likewise, your function's return value must be a specific JSON object that API Gateway can understand and convert into an HTTP response.

The Non-Proxy, or Lambda Custom Integration, pattern puts API Gateway in a more active role. Instead of just passing the request through, it can transform the request before it ever reaches your function. This is done using mapping templates written in VTL (Velocity Template Language).

This approach decouples your Lambda function from the structure of the HTTP request. Your function doesn't need to know about headers or status codes; it just receives a clean, pre-processed JSON payload tailored to its needs. You define the shape of this payload in the API Gateway console's "Integration Request" settings.

For example, you could use a VTL template to extract just the name field from an incoming JSON body and pass only that to your Lambda.

Your Lambda function now receives a much simpler event object: {"itemName": "My New Item"}. It can process this data and return a simple JSON object. API Gateway then takes this return value and, using another mapping template in the "Integration Response" settings, constructs the final HTTP response. You can map different Lambda outputs (or even errors) to different HTTP status codes, like 200 for success or 400 for a validation failure.

Choosing the right pattern depends on your use case. Lambda Proxy is often the default choice for new APIs due to its simplicity and directness. It's perfect for greenfield projects or when you want your business logic to have full control. Non-Proxy integrations shine when you're integrating with legacy systems, need to enforce a strict API contract separate from your backend logic, or want to perform simple data transformations without invoking a Lambda function at all.

Ultimately, the choice comes down to where you want to manage complexity. Do you prefer it in your function code (Proxy) or in your infrastructure configuration (Non-Proxy)? Understanding the lifecycle of a request and response in each pattern is key to building robust and maintainable serverless applications.