Loader signature changes from func() (*T, error) to
func(context.Context) (*T, error). Set.Load(ctx) already accepts a
ctx; it now flows through reload() into the loader so long-running
parses or downloads can honor ctx.Err() for graceful shutdown.
check-ip's loaders don't consume ctx yet (ipcohort/geoip are
in-memory and fast), but the hook is in place for future work.
BREAKING: dataset.Add and dataset.AddInitial signatures changed.
With --serve --async-load, blocklists + whitelist start empty and
load in background goroutines so the HTTP server binds immediately.
/healthz returns 503 until loads complete, then 200. Ignored in CLI
mode. Geo stays synchronous — geoip readers aren't nil-safe.
3,406,727 scans cleanly; 3406727 does not. Go's fmt has no
thousands-separator verb and golang.org/x/text/message pulls in a
multi-MB Unicode tree for what is 15 lines inline, so each cmd gets
its own commafy helper.
Split the stage-timing lines into a pre-stage 'Loading X... ' (shown
before the work starts) and a post-stage '<duration> (<counts>)'. Makes
it obvious something is happening during the ~1s cold-start parse
instead of going silent and printing everything at the end.
Always-on "Loading X... Nms" lines on stderr for blocklists, geoip,
and whitelist load stages. Makes it obvious at a glance that the cost
is cold-start parsing (not re-downloading) and surfaces the sizes of
the loaded sets. Stdout stays clean for pipe-friendly consumption.
os.UserCacheDir returns ~/Library/Caches on macOS, which is intended
for bundled desktop apps and hides files from anyone looking under
~/.cache. These are CLI tools — use the XDG convention everywhere so
the cache lives somewhere predictable and cross-platform-consistent.
Short-lived CLI invocations were doing a full git fetch+reset on every
run because the only debounce was an in-memory lastSynced field. MaxAge
skips the fetch when .git/FETCH_HEAD is younger than the configured
duration — git rewrites FETCH_HEAD on every successful fetch, so its
mtime is effectively "last time we talked to the remote", and it
survives process restart. Wire check-ip's blocklist repo to the same
47m refresh interval it uses for the background Tick.
Adds geoip.TarGzName(edition) as the single source of truth for the
cache filename. The _LATEST suffix signals that the file is whatever
MaxMind served most recently (versus the dated Content-Disposition
name) and keeps httpcache's ETag sidecar tied to a stable path across
releases.
Whitelist is a combined IP+CIDR cohort file polled for mtime changes;
a match short-circuits the blocklist check and marks the result
allowlisted. Drops the geoip PollFiles fallback — missing GeoIP.conf
now fails fast instead of silently polling local tarballs.
Cacher.Header is a stdlib http.Header that's merged into every request.
Authorization is stripped on redirect unconditionally (presigned S3/R2
targets, etc). Callers build the header with the usual http.Header
literal; BasicAuth/Bearer still produce the Authorization value.
The old ParseConf opened the file itself, which the name did not
convey. Now it parses the config text directly, matching
encoding/json.Unmarshal-style conventions: callers read the file (or
source the string however they like) and pass it in. Also introduce
errors.ErrMissingCredentials for the credential-missing case so callers
can branch on it.
Set handles both single-fetcher (one git repo) and multi-fetcher
(GeoLite2 City + ASN) cases uniformly. Any fetcher reporting an update
triggers a view reload. This replaces the per-caller FetcherFunc wrapper
that combined the two MaxMind cachers and the ad-hoc atomic.Pointer +
ticker goroutine in cmd/check-ip — geoip now rides on the same
Set/View/Load/Tick surface as the blocklists.
geoip now syncs via an IPCheck.Sync() method that returns (updated, err)
— same signature as gitshallow.Repo.Sync / Fetcher.Fetch. The initial
load and the background refresh goroutine both call it, so there is no
duplicated fetch+open+swap logic.
Positional args are IPs to check and print; at least one IP or --serve
must be provided. Refactor server.go: split handle into lookup + writeText
methods so main can reuse them.
geoip is no longer managed via dataset.Group — it's a single
atomic.Pointer[geoip.Databases]. The Fetcher (httpcache or PollFiles)
still drives refresh, but via an inline ticker in the serve branch that
fetches, reopens, and swaps.
Conditional Fetcher: httpcache cachers when GeoIP.conf basic auth is
present, dataset.PollFiles otherwise. geo is now a *dataset.View so the
background Tick in the serve branch refreshes it alongside blocklists.
IPCheck.ConfPath → GeoIPConfPath. After flag parsing, auto-discover the
conf path (when not explicit), parse it, and stash
cfg.GeoIPBasicAuth = httpcache.BasicAuth(...). The geoip download block
later just checks cfg.GeoIPBasicAuth != "" and uses the pre-built
value.
- httpcache.Cacher loses Transform (always atomic copy to Path); adds
BasicAuth and Bearer helpers for Authorization header values.
- geoip.Open now reads <dir>/GeoLite2-City.tar.gz and GeoLite2-ASN.tar.gz
directly: extracts the .mmdb entry in memory and opens via
geoip2.FromBytes. No .mmdb files written to disk.
- geoip.Downloader/New/NewCacher/Fetch/ExtractMMDB removed — geoip is
purely read/lookup; fetching is each caller's concern.
- cmd/check-ip/main.go is a single main() again: blocklists via
gitshallow+dataset, geoip via two httpcache.Cachers (if GeoIP.conf
present) + geoip.Open. No geo refresh loop, no dataset.Group for geo.
- cmd/geoip-update and the integration test construct httpcache.Cachers
directly against geoip.DownloadBase + edition IDs, writing .tar.gz.
Use 'GeoLite2-City.mmdb' / 'GeoLite2-ASN.mmdb' directly instead of
composing from the edition constants. Reads plainly — the actual
filename is right there.
Filenames are deterministic (<dir>/GeoLite2-City.mmdb,
<dir>/GeoLite2-ASN.mmdb) — callers no longer pass both paths. cmd/check-ip
drops its cityPath/asnPath locals and just hands the maxmind dir to
geoip.Open and the fetcher builder.
geoip.Open now just opens files; download/refresh/polling logic lives at
the cmd layer using dataset.Group with a combined httpcache.Cacher
fetcher (or PollFiles when no GeoIP.conf is available). Removes
geoip.OpenDatabases — the library is no longer concerned with refresh.
Signal handling and periodic refresh are only meaningful when the HTTP
server runs. Load blocklists with context.Background(); start Tick and
the signal-aware ctx inside the serve branch.
Follow golang-cli-flags pattern: config struct holds parsed flags and
loaded resources; handle and serve are methods on *IPCheck. Adds -V/help
pre-parse handling. Inlines clientIP into the handler.
- drop format type / formatPretty / formatJSON / requestFormat /
write / writeGeo — inline into one handler
- drop the inner check closure — inline into the handler
- one handler serves both GET / and GET /check
- fatal() replaced with log.Fatalf
- --serve is optional; without it, databases load and main returns
- drop Checker struct, loadCohort helper, and contains() nil-wrapper
- inline check logic into server as a closure
- geoip.Databases: no nil-receiver guards, no nil-field branches, no
"disabled" mode. City + ASN are both required; caller hands explicit
paths and OpenDatabases returns a fully-initialized value or an err
- main.go is now straight-line wiring with no helper functions
check-ip now takes only --serve, --geoip-conf, --blocklist-repo,
--cache-dir. Blocklist always comes from git; GeoIP mmdbs always go
through httpcache (when GeoIP.conf is available). Format negotiation
lives entirely server-side.
main.go is now straight-line wiring: parse flags, build the two
databases, run the server. All filesystem setup (MkdirAll for clone
target, for cache Path parents) is pushed into gitshallow and
httpcache so the cmd doesn't do filesystem bookkeeping.
Stats the given paths and reports updated when any size/modtime
changes since the last call. First call always reports true so the
initial Load populates views.
check-ip uses it for --inbound/--outbound so edits to local lists
get picked up by Group.Tick without a restart.
main.go now reads top-to-bottom as setup + usage of the three
databases (blocklists group, whitelist cohort, geoip readers), then
dispatch to one-shot or serve. HTTP server code moved to server.go.
No behavior change.
Libraries shouldn't decide where errors go. Tick now passes Load
errors to onError (nil to ignore); callers pick log/count/page.
check-ip supplies its own stderr writer.
geoip.Databases now exposes a structured Lookup(ip) Info. Rendering
moved up to the cmd — the library no longer writes to io.Writer.
check-ip adds a Result struct and --format flag (pretty/json). Serve
mode dispatches on ?format=json or Accept: application/json. Pretty
is the default for both one-shot and HTTP.
httpcache.Cacher.Fetch writes to <path>.tmp without MkdirAll; the
library expects the caller to own the directory. cacheDir now
MkdirAll's before returning.
Long-running server exposes GET / (client IP) and GET /check?ip= for
ad-hoc lookups. signal.NotifyContext drives graceful shutdown; the
shared dataset.Group.Tick goroutine refreshes inbound/outbound views
in the background so the refresh path gets real exercise.
Factored the shared populate+report logic into a Checker struct so
oneshot and serve modes use the same code path.
Distilled from the previous net/dataset experiment and the inline
closure version in check-ip. Keeps what actually earned its keep:
- Group ties one Fetcher to N views; a single Load drives all swaps,
so shared sources (one git pull, one zip download) don't get
re-fetched per view.
- View[T].Value() is a lock-free atomic read; the atomic.Pointer is
hidden so consumers never see in-flight reloads.
- Tick runs Load on a ticker with stderr error logging.
Dropped from the v1 design: MultiSyncer (callers fan-out inline when
needed), Close (unused outside geoip), Name (callers wrap the logger),
standalone Dataset type (Group with one view covers it), Sync vs Init
asymmetry (Load handles first-call vs update internally).
check-ip rewires to use it — file/git/http modes all build a Group
with two views, uniform shape.
Databases is now just two *geoip2.Reader fields with Open/Close/PrintInfo.
OpenDatabases still auto-discovers conf and downloads stale .mmdb files
via httpcache before opening, but it no longer runs background goroutines
or holds atomic pointers. Long-running callers that want refresh can wire
httpcache.Cacher to atomic.Pointer themselves.
check-ip drops geo.Init/geo.Run — OpenDatabases does the fetch+open work
itself, and a one-shot CLI doesn't need background refresh.
Uses atomic.Pointer[ipcohort.Cohort] directly and builds a per-source
refresh closure (files / git / http). One goroutine drives the ticker.
Exercises what the dataset pkg was abstracting so we can judge which
bits are worth a shared pkg.
Extract the file/git/httpcache mode switch into newSource and the Group
wiring into newBlocklists. main becomes flag parsing + exit code logic
only; run owns ctx and the check. Helpers (loadCohort, cacheDir,
splitCSV, loadWhitelist) are small and single-purpose.
Still exercises dataset.Group + background refresh, gitshallow, and
httpcache as before.
OpenDatabases(confPath, cityPath, asnPath) handles conf discovery, cache
dir setup, and Databases construction. DefaultConfPaths lists the standard
GeoIP.conf locations. cmd/check-ip/geo.go deleted; main calls one function.
Remove View[T] — Add now returns *Dataset[T] directly. Callers use Load()
on the returned Dataset; Init/Run belong to the owning Group.
main.go simplified: declare syncer + file paths per case, then one
g.Init() and one g.Run(). No manual loops over individual datasets.
Add gitshallow.Repo.FilePath helper.
Blocklist:
- Add -inbound, -outbound, -whitelist flags for explicit file paths
- buildSources() replaces the old constructor trio; explicit flags always win
- -data-dir and -git still work as defaults for the bitwire-it layout
GeoIP:
- Auto-discover GeoIP.conf from ./GeoIP.conf then ~/.config/maxmind/GeoIP.conf
- If no conf found and no -city-db/-asn-db given: geoip disabled silently
- If no conf but paths given: use those files (Init fails if absent)
