ErrUnexpectedStatus, ErrEmptyResponse, ErrSaveMeta are exposed so
callers can branch with errors.Is. Messages remain descriptive (status
code, URL, Path) via %w wrapping.
saveMeta now returns an error instead of silently swallowing WriteFile/
Rename failures. Fetch wraps and returns it (with updated=true, since
the body rename already succeeded). Callers get a loud signal when the
sidecar can't be written — the body is still good, but the next
conditional GET may redownload.
Cacher.Header is a stdlib http.Header that's merged into every request.
Authorization is stripped on redirect unconditionally (presigned S3/R2
targets, etc). Callers build the header with the usual http.Header
literal; BasicAuth/Bearer still produce the Authorization value.
- httpcache.Cacher loses Transform (always atomic copy to Path); adds
BasicAuth and Bearer helpers for Authorization header values.
- geoip.Open now reads <dir>/GeoLite2-City.tar.gz and GeoLite2-ASN.tar.gz
directly: extracts the .mmdb entry in memory and opens via
geoip2.FromBytes. No .mmdb files written to disk.
- geoip.Downloader/New/NewCacher/Fetch/ExtractMMDB removed — geoip is
purely read/lookup; fetching is each caller's concern.
- cmd/check-ip/main.go is a single main() again: blocklists via
gitshallow+dataset, geoip via two httpcache.Cachers (if GeoIP.conf
present) + geoip.Open. No geo refresh loop, no dataset.Group for geo.
- cmd/geoip-update and the integration test construct httpcache.Cachers
directly against geoip.DownloadBase + edition IDs, writing .tar.gz.
check-ip now takes only --serve, --geoip-conf, --blocklist-repo,
--cache-dir. Blocklist always comes from git; GeoIP mmdbs always go
through httpcache (when GeoIP.conf is available). Format negotiation
lives entirely server-side.
main.go is now straight-line wiring: parse flags, build the two
databases, run the server. All filesystem setup (MkdirAll for clone
target, for cache Path parents) is pushed into gitshallow and
httpcache so the cmd doesn't do filesystem bookkeeping.
- Each package gets its own go.mod: net/{dataset,httpcache,gitshallow,ipcohort,geoip,formmailer}
- go.work with replace directives for cross-module workspace resolution
- dataset.Syncer/NopSyncer moved here from httpcache; callers duck-type it
- dataset.View[T] returned by Add to prevent Init/Sync/Run misuse on group members
- cmd/check-ip moved from net/ipcohort/cmd/check-ip to top-level cmd/check-ip
- Add net/ipcohort/cmd/ipcohort-contains for standalone cohort membership testing
httpcache: write <path>.meta JSON sidecar after each successful download;
load it on first Fetch so conditional GETs work after process restarts.
Tests verify: download, sidecar written, same-cacher 304, fresh-cacher 304
(the last being the key case — no in-memory state, sidecar drives ETag).
MaxMind integration test reads GeoIP.conf, downloads City+ASN, verifies
fresh-cacher conditional GET skips re-download via sidecar ETag.
Generic header pair works for any auth scheme — Bearer, X-API-Key, Basic, etc.
Auth is forwarded on redirects; the MaxMind-specific stripping is removed.
geoip.go encodes Basic auth credentials directly into AuthValue.
ConnTimeout (default 5s) caps TCP connect + TLS handshake via net.Dialer
and Transport.TLSHandshakeTimeout. Timeout (default 5m) caps the overall
request including body read. Previously a single 30s timeout covered both,
which was too short for large downloads and too long for connection failures.
Sources.Init() was redundant: gitshallow.Repo.Fetch() already clones
if missing via syncGit()->clone(). Removing it means blGroup.Init()
is the single entry point, no duplicate network calls.
httpcache.NopSyncer{} replaces the private nopSyncer in the cmd —
exported so any caller can build a file-only Dataset without a syncer.
httpcache.Syncer interface: Fetch() (bool, error) — satisfied by both
*httpcache.Cacher and *gitshallow.Repo (new Fetch method + LightGC field).
httpcache.Cacher.Fetch now errors on zero-length 200 response instead of
clobbering the existing file with empty content.
Sources.Fetch/Init drop the lightGC param (baked into Repo.LightGC).
Sources.syncs []httpcache.Syncer replaces the separate git/httpInbound/
httpOutbound fields — Fetch iterates syncs uniformly, no more switch.
Sources itself satisfies httpcache.Syncer.
httpcache.Cacher gains:
- Username/Password: Basic Auth, stripped before following redirects
- MaxAge: skip HTTP if local file mtime is within this duration
- MinInterval: skip HTTP if last Fetch attempt was within this duration
- Transform: post-process response body (e.g. extract .mmdb from tar.gz)
geoip.Downloader now builds an httpcache.Cacher via NewCacher(), removing
its own HTTP client. ExtractMMDB is now exported for use as a Transform.
check-ip-blacklist renamed to check-ip; adds -city-db / -asn-db flags
for GeoLite2 lookup (country, city, subdivision, ASN) printed after each
blocklist result.
Top-layer callers (IPFilter) now drive all reloads directly after
Sync/Fetch return. gitshallow.Init now returns (bool, error).
httpcache drops Init and Sync — callers just call Fetch.
Blacklist → IPFilter with three separate atomic cohorts: whitelist
(never blocked), inbound, and outbound. ContainsInbound/ContainsOutbound
each skip the whitelist. HTTP sync fetches all cachers before a single
reload to avoid double-load. Also fixes httpcache.Init calling c.Fetch().
