Propagate the patterns used in cmd/check-ip to the other command-line
tools touched by this PR:
- flag.FlagSet + Config struct instead of package-level flag.String
pointers (geoip-update, ipcohort-contains, git-shallow-sync).
- -V/--version/version and help/-help/--help handled before Parse,
matching the project's CLI conventions.
- Stderr "Loading X... Nms (counts)" progress lines on the stages that
actually take time: blocklist cohort parse (ipcohort-contains),
per-edition fetch (geoip-update), and repo sync (git-shallow-sync).
Stdout stays machine-parseable.
os.UserCacheDir returns ~/Library/Caches on macOS, which is intended
for bundled desktop apps and hides files from anyone looking under
~/.cache. These are CLI tools — use the XDG convention everywhere so
the cache lives somewhere predictable and cross-platform-consistent.
Prefer <edition>_LATEST.tar.gz (what httpcache writes), but fall back
to the lexicographically greatest <edition>_*.tar.gz — MaxMind's dated
Content-Disposition names sort chronologically, so this picks the most
recent archive when the cache was populated by hand or by another tool.
Exposes FindTarGz for callers that need the resolved path.
Adds geoip.TarGzName(edition) as the single source of truth for the
cache filename. The _LATEST suffix signals that the file is whatever
MaxMind served most recently (versus the dated Content-Disposition
name) and keeps httpcache's ETag sidecar tied to a stable path across
releases.
Cacher.Header is a stdlib http.Header that's merged into every request.
Authorization is stripped on redirect unconditionally (presigned S3/R2
targets, etc). Callers build the header with the usual http.Header
literal; BasicAuth/Bearer still produce the Authorization value.
The old ParseConf opened the file itself, which the name did not
convey. Now it parses the config text directly, matching
encoding/json.Unmarshal-style conventions: callers read the file (or
source the string however they like) and pass it in. Also introduce
errors.ErrMissingCredentials for the credential-missing case so callers
can branch on it.
- httpcache.Cacher loses Transform (always atomic copy to Path); adds
BasicAuth and Bearer helpers for Authorization header values.
- geoip.Open now reads <dir>/GeoLite2-City.tar.gz and GeoLite2-ASN.tar.gz
directly: extracts the .mmdb entry in memory and opens via
geoip2.FromBytes. No .mmdb files written to disk.
- geoip.Downloader/New/NewCacher/Fetch/ExtractMMDB removed — geoip is
purely read/lookup; fetching is each caller's concern.
- cmd/check-ip/main.go is a single main() again: blocklists via
gitshallow+dataset, geoip via two httpcache.Cachers (if GeoIP.conf
present) + geoip.Open. No geo refresh loop, no dataset.Group for geo.
- cmd/geoip-update and the integration test construct httpcache.Cachers
directly against geoip.DownloadBase + edition IDs, writing .tar.gz.
Use 'GeoLite2-City.mmdb' / 'GeoLite2-ASN.mmdb' directly instead of
composing from the edition constants. Reads plainly — the actual
filename is right there.
Filenames are deterministic (<dir>/GeoLite2-City.mmdb,
<dir>/GeoLite2-ASN.mmdb) — callers no longer pass both paths. cmd/check-ip
drops its cityPath/asnPath locals and just hands the maxmind dir to
geoip.Open and the fetcher builder.
geoip.Open now just opens files; download/refresh/polling logic lives at
the cmd layer using dataset.Group with a combined httpcache.Cacher
fetcher (or PollFiles when no GeoIP.conf is available). Removes
geoip.OpenDatabases — the library is no longer concerned with refresh.
- drop Checker struct, loadCohort helper, and contains() nil-wrapper
- inline check logic into server as a closure
- geoip.Databases: no nil-receiver guards, no nil-field branches, no
"disabled" mode. City + ASN are both required; caller hands explicit
paths and OpenDatabases returns a fully-initialized value or an err
- main.go is now straight-line wiring with no helper functions
geoip.Databases now exposes a structured Lookup(ip) Info. Rendering
moved up to the cmd — the library no longer writes to io.Writer.
check-ip adds a Result struct and --format flag (pretty/json). Serve
mode dispatches on ?format=json or Accept: application/json. Pretty
is the default for both one-shot and HTTP.
Databases is now just two *geoip2.Reader fields with Open/Close/PrintInfo.
OpenDatabases still auto-discovers conf and downloads stale .mmdb files
via httpcache before opening, but it no longer runs background goroutines
or holds atomic pointers. Long-running callers that want refresh can wire
httpcache.Cacher to atomic.Pointer themselves.
check-ip drops geo.Init/geo.Run — OpenDatabases does the fetch+open work
itself, and a one-shot CLI doesn't need background refresh.
OpenDatabases(confPath, cityPath, asnPath) handles conf discovery, cache
dir setup, and Databases construction. DefaultConfPaths lists the standard
GeoIP.conf locations. cmd/check-ip/geo.go deleted; main calls one function.
- Each package gets its own go.mod: net/{dataset,httpcache,gitshallow,ipcohort,geoip,formmailer}
- go.work with replace directives for cross-module workspace resolution
- dataset.Syncer/NopSyncer moved here from httpcache; callers duck-type it
- dataset.View[T] returned by Add to prevent Init/Sync/Run misuse on group members
- cmd/check-ip moved from net/ipcohort/cmd/check-ip to top-level cmd/check-ip
- Add net/ipcohort/cmd/ipcohort-contains for standalone cohort membership testing
- geoip.DefaultCacheDir() → ~/.cache/maxmind (os.UserCacheDir based)
- check-ip defaults data dir to ~/.cache/bitwire-it; -data-dir flag overrides;
positional data-dir arg removed (IP is now the only required arg)
- geoip conf: DatabaseDirectory defaults to geoip.DefaultCacheDir() when blank
- httpcache integration tests now cover both inbound files (single_ips + networks)
httpcache: write <path>.meta JSON sidecar after each successful download;
load it on first Fetch so conditional GETs work after process restarts.
Tests verify: download, sidecar written, same-cacher 304, fresh-cacher 304
(the last being the key case — no in-memory state, sidecar drives ETag).
MaxMind integration test reads GeoIP.conf, downloads City+ASN, verifies
fresh-cacher conditional GET skips re-download via sidecar ETag.
Generic header pair works for any auth scheme — Bearer, X-API-Key, Basic, etc.
Auth is forwarded on redirects; the MaxMind-specific stripping is removed.
geoip.go encodes Basic auth credentials directly into AuthValue.
geoip.ParseConf() extracted from geoip-update into the geoip package so
both cmds can read GeoIP.conf without duplication.
check-ip gains -geoip-conf flag: reads AccountID+LicenseKey, resolves
mmdb paths into data-dir, builds httpcache.Cachers with geoip.NewCacher.
Background runLoop now refreshes both blocklists and GeoIP DBs on each
tick, hot-swapping geoip2.Reader via atomic.Pointer.Swap + old.Close().
httpcache.Cacher gains:
- Username/Password: Basic Auth, stripped before following redirects
- MaxAge: skip HTTP if local file mtime is within this duration
- MinInterval: skip HTTP if last Fetch attempt was within this duration
- Transform: post-process response body (e.g. extract .mmdb from tar.gz)
geoip.Downloader now builds an httpcache.Cacher via NewCacher(), removing
its own HTTP client. ExtractMMDB is now exported for use as a Transform.
check-ip-blacklist renamed to check-ip; adds -city-db / -asn-db flags
for GeoLite2 lookup (country, city, subdivision, ASN) printed after each
blocklist result.
