S3 without an SDK

rcp copy /data s3://backups/site-a -s3.endpoint http://minio.lab:9000 speaks the S3 REST API with AWS Signature Version 4, over a small HTTP/1.1 client in the standard library: ListObjectsV2, HeadObject, GetObject (with ranges), PutObject, multipart uploads, CopyObject and DeleteObject. Credentials and region resolve the way the AWS tools resolve them: AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY / AWS_SESSION_TOKEN / AWS_REGION first, then the profile named by -s3.profile or AWS_PROFILE in ~/.aws/credentials and ~/.aws/config (including its endpoint_url and region). -s3.endpoint URL (or AWS_ENDPOINT_URL) names a MinIO, ONTAP S3 or other service, addressed path-style; with no endpoint the bucket is on AWS, addressed virtual-hosted in its region. A -s3.endpoint carrying user:password@ is refused by name, as the WebDAV and http endpoints are and for the same reason, and the refusal does not repeat the credential – an endpoint reaches the job log, a report and ps, so a message that quotes it back writes the password into the log on the one path that exists to keep it out. -s3.insecure picks plain HTTP when the endpoint names no scheme, -s3.noverify skips certificate verification (and says so), AWS_CA_BUNDLE names a private CA, and HTTPS_PROXY / HTTP_PROXY / NO_PROXY are honored, CONNECT for TLS and absolute URIs for plain HTTP.

A bucket has no inodes, so the tree is spelled out the way s3fs spells it: dir/ zero-byte objects are directories (a directory that only exists as a key prefix is listed too), a symlink is an object whose body is its target, and modification time, mode, uid and gid ride in user metadata (x-amz-meta-mtime as seconds.nanoseconds, mode as the full st_mode in decimal, uid, gid). A tree copied into a bucket and back comes home with its times, modes and owners; verify between a directory and a bucket compares all of them. Hard links have no representation: the second name becomes a server-side copy (no data moves), verify reports the lost link unless -nolinks, and extended attributes are reported as refused.

Uploads stream. The executor declares each file’s size up front, so an object up to 64 MiB is one PUT with a known Content-Length and a larger one is a multipart upload whose parts open as the bytes arrive, sized so that any object fits in the service’s 10,000 parts; a sparse source’s holes are written as zeros because objects have none. Nothing is buffered beyond the socket, whatever the object size. Reads are one streaming GET per file, reissued with a Range on a seek. Listing is ListObjectsV2 in pages of 1,000 followed by one HEAD per object in parallel for the metadata, which is the honest cost of carrying POSIX metadata in a bucket; scan -q on a million objects is a million small requests.

TLS is the one thing not in the standard library. It is behind the rustls cargo feature (rustls with the ring provider), so the default build carries no TLS stack and speaks plain HTTP, which is what a MinIO on a kit or a gateway on localhost offers, and refuses an https:// endpoint with a message naming the feature. cargo build --release --features rustls needs the registry once (the lockfile pins the versions) and produces a binary that speaks to AWS and to a MinIO behind a private CA.

The contract suite runs the client against an in-process stand-in service (src/s3/testserver.rs) that verifies every request’s signature and payload hash, so a signing mistake fails a test here; the signer is also checked against the worked examples in the S3 developer guide. A real service (AWS, MinIO, ONTAP) is a separate check to make before trusting a migration to it.