Whitelist is a combined IP+CIDR cohort file polled for mtime changes;
a match short-circuits the blocklist check and marks the result
allowlisted. Drops the geoip PollFiles fallback — missing GeoIP.conf
now fails fast instead of silently polling local tarballs.
Cacher.Header is a stdlib http.Header that's merged into every request.
Authorization is stripped on redirect unconditionally (presigned S3/R2
targets, etc). Callers build the header with the usual http.Header
literal; BasicAuth/Bearer still produce the Authorization value.
The old ParseConf opened the file itself, which the name did not
convey. Now it parses the config text directly, matching
encoding/json.Unmarshal-style conventions: callers read the file (or
source the string however they like) and pass it in. Also introduce
errors.ErrMissingCredentials for the credential-missing case so callers
can branch on it.
Set handles both single-fetcher (one git repo) and multi-fetcher
(GeoLite2 City + ASN) cases uniformly. Any fetcher reporting an update
triggers a view reload. This replaces the per-caller FetcherFunc wrapper
that combined the two MaxMind cachers and the ad-hoc atomic.Pointer +
ticker goroutine in cmd/check-ip — geoip now rides on the same
Set/View/Load/Tick surface as the blocklists.
geoip now syncs via an IPCheck.Sync() method that returns (updated, err)
— same signature as gitshallow.Repo.Sync / Fetcher.Fetch. The initial
load and the background refresh goroutine both call it, so there is no
duplicated fetch+open+swap logic.
Positional args are IPs to check and print; at least one IP or --serve
must be provided. Refactor server.go: split handle into lookup + writeText
methods so main can reuse them.
geoip is no longer managed via dataset.Group — it's a single
atomic.Pointer[geoip.Databases]. The Fetcher (httpcache or PollFiles)
still drives refresh, but via an inline ticker in the serve branch that
fetches, reopens, and swaps.
Conditional Fetcher: httpcache cachers when GeoIP.conf basic auth is
present, dataset.PollFiles otherwise. geo is now a *dataset.View so the
background Tick in the serve branch refreshes it alongside blocklists.
IPCheck.ConfPath → GeoIPConfPath. After flag parsing, auto-discover the
conf path (when not explicit), parse it, and stash
cfg.GeoIPBasicAuth = httpcache.BasicAuth(...). The geoip download block
later just checks cfg.GeoIPBasicAuth != "" and uses the pre-built
value.
- httpcache.Cacher loses Transform (always atomic copy to Path); adds
BasicAuth and Bearer helpers for Authorization header values.
- geoip.Open now reads <dir>/GeoLite2-City.tar.gz and GeoLite2-ASN.tar.gz
directly: extracts the .mmdb entry in memory and opens via
geoip2.FromBytes. No .mmdb files written to disk.
- geoip.Downloader/New/NewCacher/Fetch/ExtractMMDB removed — geoip is
purely read/lookup; fetching is each caller's concern.
- cmd/check-ip/main.go is a single main() again: blocklists via
gitshallow+dataset, geoip via two httpcache.Cachers (if GeoIP.conf
present) + geoip.Open. No geo refresh loop, no dataset.Group for geo.
- cmd/geoip-update and the integration test construct httpcache.Cachers
directly against geoip.DownloadBase + edition IDs, writing .tar.gz.
Use 'GeoLite2-City.mmdb' / 'GeoLite2-ASN.mmdb' directly instead of
composing from the edition constants. Reads plainly — the actual
filename is right there.
Filenames are deterministic (<dir>/GeoLite2-City.mmdb,
<dir>/GeoLite2-ASN.mmdb) — callers no longer pass both paths. cmd/check-ip
drops its cityPath/asnPath locals and just hands the maxmind dir to
geoip.Open and the fetcher builder.
geoip.Open now just opens files; download/refresh/polling logic lives at
the cmd layer using dataset.Group with a combined httpcache.Cacher
fetcher (or PollFiles when no GeoIP.conf is available). Removes
geoip.OpenDatabases — the library is no longer concerned with refresh.
Signal handling and periodic refresh are only meaningful when the HTTP
server runs. Load blocklists with context.Background(); start Tick and
the signal-aware ctx inside the serve branch.
Follow golang-cli-flags pattern: config struct holds parsed flags and
loaded resources; handle and serve are methods on *IPCheck. Adds -V/help
pre-parse handling. Inlines clientIP into the handler.
- drop format type / formatPretty / formatJSON / requestFormat /
write / writeGeo — inline into one handler
- drop the inner check closure — inline into the handler
- one handler serves both GET / and GET /check
- fatal() replaced with log.Fatalf
- --serve is optional; without it, databases load and main returns
- drop Checker struct, loadCohort helper, and contains() nil-wrapper
- inline check logic into server as a closure
- geoip.Databases: no nil-receiver guards, no nil-field branches, no
"disabled" mode. City + ASN are both required; caller hands explicit
paths and OpenDatabases returns a fully-initialized value or an err
- main.go is now straight-line wiring with no helper functions
check-ip now takes only --serve, --geoip-conf, --blocklist-repo,
--cache-dir. Blocklist always comes from git; GeoIP mmdbs always go
through httpcache (when GeoIP.conf is available). Format negotiation
lives entirely server-side.
main.go is now straight-line wiring: parse flags, build the two
databases, run the server. All filesystem setup (MkdirAll for clone
target, for cache Path parents) is pushed into gitshallow and
httpcache so the cmd doesn't do filesystem bookkeeping.
Stats the given paths and reports updated when any size/modtime
changes since the last call. First call always reports true so the
initial Load populates views.
check-ip uses it for --inbound/--outbound so edits to local lists
get picked up by Group.Tick without a restart.
main.go now reads top-to-bottom as setup + usage of the three
databases (blocklists group, whitelist cohort, geoip readers), then
dispatch to one-shot or serve. HTTP server code moved to server.go.
No behavior change.
Libraries shouldn't decide where errors go. Tick now passes Load
errors to onError (nil to ignore); callers pick log/count/page.
check-ip supplies its own stderr writer.
geoip.Databases now exposes a structured Lookup(ip) Info. Rendering
moved up to the cmd — the library no longer writes to io.Writer.
check-ip adds a Result struct and --format flag (pretty/json). Serve
mode dispatches on ?format=json or Accept: application/json. Pretty
is the default for both one-shot and HTTP.
httpcache.Cacher.Fetch writes to <path>.tmp without MkdirAll; the
library expects the caller to own the directory. cacheDir now
MkdirAll's before returning.
Long-running server exposes GET / (client IP) and GET /check?ip= for
ad-hoc lookups. signal.NotifyContext drives graceful shutdown; the
shared dataset.Group.Tick goroutine refreshes inbound/outbound views
in the background so the refresh path gets real exercise.
Factored the shared populate+report logic into a Checker struct so
oneshot and serve modes use the same code path.
Distilled from the previous net/dataset experiment and the inline
closure version in check-ip. Keeps what actually earned its keep:
- Group ties one Fetcher to N views; a single Load drives all swaps,
so shared sources (one git pull, one zip download) don't get
re-fetched per view.
- View[T].Value() is a lock-free atomic read; the atomic.Pointer is
hidden so consumers never see in-flight reloads.
- Tick runs Load on a ticker with stderr error logging.
Dropped from the v1 design: MultiSyncer (callers fan-out inline when
needed), Close (unused outside geoip), Name (callers wrap the logger),
standalone Dataset type (Group with one view covers it), Sync vs Init
asymmetry (Load handles first-call vs update internally).
check-ip rewires to use it — file/git/http modes all build a Group
with two views, uniform shape.
Databases is now just two *geoip2.Reader fields with Open/Close/PrintInfo.
OpenDatabases still auto-discovers conf and downloads stale .mmdb files
via httpcache before opening, but it no longer runs background goroutines
or holds atomic pointers. Long-running callers that want refresh can wire
httpcache.Cacher to atomic.Pointer themselves.
check-ip drops geo.Init/geo.Run — OpenDatabases does the fetch+open work
itself, and a one-shot CLI doesn't need background refresh.
Uses atomic.Pointer[ipcohort.Cohort] directly and builds a per-source
refresh closure (files / git / http). One goroutine drives the ticker.
Exercises what the dataset pkg was abstracting so we can judge which
bits are worth a shared pkg.
Extract the file/git/httpcache mode switch into newSource and the Group
wiring into newBlocklists. main becomes flag parsing + exit code logic
only; run owns ctx and the check. Helpers (loadCohort, cacheDir,
splitCSV, loadWhitelist) are small and single-purpose.
Still exercises dataset.Group + background refresh, gitshallow, and
httpcache as before.
OpenDatabases(confPath, cityPath, asnPath) handles conf discovery, cache
dir setup, and Databases construction. DefaultConfPaths lists the standard
GeoIP.conf locations. cmd/check-ip/geo.go deleted; main calls one function.
Remove View[T] — Add now returns *Dataset[T] directly. Callers use Load()
on the returned Dataset; Init/Run belong to the owning Group.
main.go simplified: declare syncer + file paths per case, then one
g.Init() and one g.Run(). No manual loops over individual datasets.
Add gitshallow.Repo.FilePath helper.
Blocklist:
- Add -inbound, -outbound, -whitelist flags for explicit file paths
- buildSources() replaces the old constructor trio; explicit flags always win
- -data-dir and -git still work as defaults for the bitwire-it layout
GeoIP:
- Auto-discover GeoIP.conf from ./GeoIP.conf then ~/.config/maxmind/GeoIP.conf
- If no conf found and no -city-db/-asn-db given: geoip disabled silently
- If no conf but paths given: use those files (Init fails if absent)
- Each package gets its own go.mod: net/{dataset,httpcache,gitshallow,ipcohort,geoip,formmailer}
- go.work with replace directives for cross-module workspace resolution
- dataset.Syncer/NopSyncer moved here from httpcache; callers duck-type it
- dataset.View[T] returned by Add to prevent Init/Sync/Run misuse on group members
- cmd/check-ip moved from net/ipcohort/cmd/check-ip to top-level cmd/check-ip
- Add net/ipcohort/cmd/ipcohort-contains for standalone cohort membership testing
Replace directives pointing to local paths prevent `go install` from
working. Use published module versions (sqlmigrate v1.0.2, shmigrate
v1.0.2) so users can install with:
go install github.com/therootcompany/golib/cmd/sql-migrate/v2@latest
- Use full flag names in command defaults (--no-align, --tuples-only, etc.)
- Add sqlite/sqlite3/lite alias for SQLite
- Add sqlcmd/mssql/sqlserver alias for SQL Server (go-sqlcmd, TDS 8.0)
- Add SQL Server help text with SQLCMD env vars and TLS docs
- Add sqlite and sqlcmd SELECT expressions for id+name migration log
Migration{ID, Name} is the identity type returned by Up/Down/Latest/Drop.
Script{Migration, Up, Down} holds collected SQL content from Collect().
Migrator interface now takes SQL as a separate parameter:
ExecUp(ctx, Migration, sql string)
ExecDown(ctx, Migration, sql string)
This separates identity from content — callers that track what ran
don't need to carry around SQL strings they'll never use.
Updates shmigrate to match (ignores the sql parameter, references
files on disk instead).
- logMigrationsSelect() returns DB-specific SELECT for id+name output,
matching psql explicitly and erroring on unrecognized --sql-command
- logMigrationsQueryNote const for the comment header
- maybeUpgradeLogQuery() auto-upgrades old _migrations.sql on first run,
replacing only the matching SELECT line
- Add UPGRADING help section pointing to sync subcommand
