Skip to main content

Documentation Index

Fetch the complete documentation index at: https://mintlify.com/sachnun/hugbucket/llms.txt

Use this file to discover all available pages before exploring further.

HugBucket is configured entirely through environment variables. The table below is a quick-reference summary; detailed descriptions follow.

Quick reference

VariableRequiredDefaultDescription
MODEYesRun mode: s3 or ftp
HF_TOKENYesHugging Face API token
AWS_ACCESS_KEY_IDNo""S3 access key
AWS_SECRET_ACCESS_KEYNo""S3 secret key
FTP_USERNAMENo""FTP username (empty = anonymous)
FTP_PASSWORDNo""FTP password (empty = anonymous)

Required variables

MODE
string
required
Selects the gateway protocol. Must be either s3 or ftp. The process exits with an error if this variable is missing or empty.
HF_TOKEN
string
required
Your Hugging Face API token. Used to authenticate all requests to the Hugging Face storage backend and to resolve your namespace via /api/whoami-v2 at startup.
Generate a token with write access at huggingface.co/settings/tokens.

S3 authentication

These variables apply only when MODE=s3. They define the credentials that S3 clients must present when connecting to the HugBucket S3 endpoint.
AWS_ACCESS_KEY_ID
string
default:""
The access key ID that S3 clients must supply. Maps to the Hugging Face token internally.
AWS_SECRET_ACCESS_KEY
string
default:""
The secret access key that S3 clients must supply alongside AWS_ACCESS_KEY_ID.
If both AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY are left empty, S3 authentication is disabled — any client can connect without credentials. Set both to non-empty values in production.
Setting only one of the two S3 credentials does not produce an error but will leave the gateway in an inconsistent authentication state. Always set both or neither.

FTP settings

These variables apply only when MODE=ftp.
FTP_USERNAME
string
default:""
The FTP username clients must authenticate with. Must be set together with FTP_PASSWORD.
FTP_PASSWORD
string
default:""
The FTP password clients must authenticate with. Must be set together with FTP_USERNAME.
Leave both FTP_USERNAME and FTP_PASSWORD empty to allow anonymous FTP access. Setting only one of the two is a configuration error and will cause the server to refuse to start.

Internal configuration defaults

The following settings are defined in the Config dataclass (hugbucket/config.py) and cannot currently be overridden via environment variables. They are documented here for reference.

Network

SettingDefaultDescription
host0.0.0.0S3 gateway bind address
port9000S3 gateway bind port
ftp_host0.0.0.0FTP gateway bind address
ftp_port2121FTP gateway bind port
regionus-east-1S3 region advertised to clients
hf_endpointhttps://huggingface.coHugging Face API base URL
http_pool_size0Max outbound connections (0 = unlimited)

Xet CDC (content-defined chunking)

SettingDefaultDescription
xet_chunk_target65536 (64 KiB)Target chunk size
xet_chunk_min8192 (8 KiB)Minimum chunk size
xet_chunk_max131072 (128 KiB)Maximum chunk size
xet_xorb_max_bytes67108864 (64 MiB)Maximum XORB size

Upload

SettingDefaultDescription
cas_upload_timeout300Seconds before a CAS request times out
cas_upload_retries3Retry attempts for CAS xorb/shard uploads
cas_retry_base_delay1.0Base delay in seconds for exponential backoff
multipart_upload_ttl86400 (24 h)Seconds before stale multipart uploads are cleaned up

Cache

SettingDefaultDescription
xorb_cache_max_bytes536870912 (512 MiB)Maximum in-memory XORB cache size
recon_cache_max_entries1024Maximum reconstruction cache entries
recon_cache_ttl300 (5 min)Reconstruction cache entry TTL in seconds
file_info_cache_max_entries256Maximum file info cache entries
file_info_cache_ttl30 (30 s)File info cache TTL in seconds — kept short for consistency