• Receive the file as multipart/form-data on your API server
• Write the file temporarily to the local filesystem,
• Push a job onto a queue, and let a worker ship the file off to S3.

The server is just acting as a relay for bytes. The application server has bounded memory, shared I/O, and a finite pool of request handlers. It is a poor relay for large binary payloads.

The breaking point is usually a specific combination: large files, multiple users uploading simultaneously, and a server already doing other work.

Anatomy of a multipart upload

Over the wire:

When a browser or HTTP client submits a file viamultipart/form-dataThe request body is split into named parts, each separated by a randomly generated boundary string. A minimal upload of a PDF looks like this on the wire:

POST /upload HTTP/1.1 Host: api.example.com Content-Type: multipart/form-data; boundary= - - FormBoundaryXk3Rp9 Content-Length: 2097152 Transfer-Encoding: chunked - - - FormBoundaryXk3Rp9 Content-Disposition: form-data; name="file"; filename="something.pdf" Content-Type: application/pdf %PDF-1.4 … [raw binary payload - every byte of the file] … - - - FormBoundaryXk3Rp9 -

The bytes themselves travel over the wire from the client to your server as part of the HTTP request.

What your framework does with it

1. TCP packets arrive at your server in chunks. The OS network stack assembles them into a stream. The HTTP server (Node's http module, uvicorn, etc.) reads this stream and begins feeding it to the body parser.
2. The parser buffers into memory or spools to disk
3. After the upload is fully received, the parser hands control to the route handler, which points to either the in-memory buffer or the temp path on disk.

What your handler does with it

1. The AWS SDK opens a second I/O channel and re-transmits every byte your server just received.
2. The temp file is deleted.
3. The response is sent.

Every byte makes two trips: client → server, then server → S3. For a 2 MB PDF, that's 4 MB of I/O your server is doing purely as a pipe.

The Streaming Illusion

We can pipe the inbound multipart stream directly to cloud object storage without touching disk. The stream is passed through in chunks, so memory consumption per upload stays low and roughly constant regardless of file size.

But streaming does not eliminate the server as a bottleneck. 30 concurrent 2 MB uploads means your server is pushing up to 60 MB of outbound transfer to S3 simultaneously — bandwidth that comes directly out of your instance budget. You're also holding one active I/O handle pair per upload — one reading from the client, one writing to S3 — for the full duration of the transfer. Furthermore, if S3 ingestion slows down, backpressure builds. Your server has to pause the inbound stream, holding state and tying up worker threads while it waits.

The Pivot: Presigned URLs

A presigned URL is a time-limited, cryptographically signed S3 URL that grants a specific HTTP operation, PUT in our case, to any bearer, without requiring AWS credentials. Generate it on the server, hand it to the client, and have the client upload it directly to S3. Your application server is never in the data path.

Implementation

# S3 client setup

import boto3
from botocore.config import Config


s3_client = boto3.client(
  "s3",
  aws_access_key_id="aws_access_key_id",
  aws_secret_access_key="aws_secret_access_key",
  region_name="region_name",
  config=Config(
    retries={
      "mode": "adaptive",
      "max_attempts": 3,
    },
  )
)


def generate_presigned_url(
  client_method: str,
  params: dict[str, str],
  expires_in: int,
) -> str:
  try:
    return s3_client.generate_presigned_url(
        clientmethod=client_method,
        params=params,
        expiresin=expires_in
    )
  except ClientError as e:
    raise e

# Validation Schemas

from dataclasses import dataclass

from fastapi import HTTPException, status

VALID_FILE_FORMATS = {
    "application/pdf",
    "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
}
MAX_FILE_SIZE = 2 * 1024 * 1024  # 2MB

@dataclass
class FileUploadSchema:
    filename: str
    size: int
    content_type: str

    def __post_init__(self):
        if not self.filename or not self.filename.strip():
            raise HTTPException(
                status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
                detail="Filename is required",
            )
        if self.content_type not in VALID_FILE_FORMATS:
            raise HTTPException(
                status_code=status.HTTP_422_UNPROCESSABLE_ENTITY,
                detail="Invalid file format. Only PDF and DOCX are allowed.",
            )
        if self.size and self.size > MAX_FILE_SIZE:
            raise HTTPException(
                status_code=status.HTTP_413_REQUEST_ENTITY_TOO_LARGE,
                detail="File size exceeds the maximum limit of 2MB.",
            )

# API Endpoint

@router.post("/uploads/presign")
async def generate_presigned_url(payload: FileUploadSchema):
    url = generate_presigned_url(
        client_method="put_object",
        params={
            "Bucket": "Bucket",
            "Key": "Key",
            "ContentType": payload.content_type,
        },
        expires_in=settings.presigned_url_ttl,
    )

    return {"upload_url": url}

Why presigned URLs win?

• Zero server buffering: File bytes travel directly from the client to S3.
• Horizontal scale for free: S3 handles thousands of concurrent large uploads natively.
• Faster uploads for users: Clients connect directly to S3's edge. AWS's global infrastructure is often closer than your server.
• Simpler architecture: No queue, no worker process, no temp-file cleanup job.
• Built-in expiry and scope: Each URL is valid for exactly one operation on exactly one key, for a bounded time window. Leaked URLs are self-limiting without any action on your part.

Security Considerations

With presigned URLs, you delegate a PUT operation directly to the client. That delegation is cryptographically scoped and time-limited, but it still means S3 will accept whatever bytes the client sends, as long as the signature is valid.

What the signature actually enforces

✓ Bucket and key: The client cannot change the destination.

✓ HTTP method: A put_object presigned URL cannot be used to delete, list, or read.

✓ Content-Type (if included): The client must send that exact header, or S3 rejects the request.

✓ Expiry window: The URL is valid only until ExpiresIn seconds have elapsed. After that, S3 returns 403 regardless of signature validity.

What the signature bypasses

✗ File contents: A valid presigned URL for application/pdf will accept a malicious payload dressed as a PDF. S3 does not inspect content.

✗ File size: A basic PutObject presigned URL has no size constraint. A client can upload a 2 GB file to a URL you intended for a 2 MB file.

When to stick with the old approach

• When you need to parse or validate file contents before storage (e.g., rejecting malformed CSVs)
• When compliance requires data to pass through an audited network boundary, or
• When you are writing an internal tool, simplicity matters more than scale.

Summary

Presigned URLs shift the upload burden from your infrastructure to S3. The API surface is small

• One endpoint to generate the URL
• One to confirm completion: no buffering, no workers, lower cost, and a faster experience.