main.go now reads top-to-bottom as setup + usage of the three
databases (blocklists group, whitelist cohort, geoip readers), then
dispatch to one-shot or serve. HTTP server code moved to server.go.
No behavior change.
Libraries shouldn't decide where errors go. Tick now passes Load
errors to onError (nil to ignore); callers pick log/count/page.
check-ip supplies its own stderr writer.
geoip.Databases now exposes a structured Lookup(ip) Info. Rendering
moved up to the cmd — the library no longer writes to io.Writer.
check-ip adds a Result struct and --format flag (pretty/json). Serve
mode dispatches on ?format=json or Accept: application/json. Pretty
is the default for both one-shot and HTTP.
httpcache.Cacher.Fetch writes to <path>.tmp without MkdirAll; the
library expects the caller to own the directory. cacheDir now
MkdirAll's before returning.
Long-running server exposes GET / (client IP) and GET /check?ip= for
ad-hoc lookups. signal.NotifyContext drives graceful shutdown; the
shared dataset.Group.Tick goroutine refreshes inbound/outbound views
in the background so the refresh path gets real exercise.
Factored the shared populate+report logic into a Checker struct so
oneshot and serve modes use the same code path.
Distilled from the previous net/dataset experiment and the inline
closure version in check-ip. Keeps what actually earned its keep:
- Group ties one Fetcher to N views; a single Load drives all swaps,
so shared sources (one git pull, one zip download) don't get
re-fetched per view.
- View[T].Value() is a lock-free atomic read; the atomic.Pointer is
hidden so consumers never see in-flight reloads.
- Tick runs Load on a ticker with stderr error logging.
Dropped from the v1 design: MultiSyncer (callers fan-out inline when
needed), Close (unused outside geoip), Name (callers wrap the logger),
standalone Dataset type (Group with one view covers it), Sync vs Init
asymmetry (Load handles first-call vs update internally).
check-ip rewires to use it — file/git/http modes all build a Group
with two views, uniform shape.
Databases is now just two *geoip2.Reader fields with Open/Close/PrintInfo.
OpenDatabases still auto-discovers conf and downloads stale .mmdb files
via httpcache before opening, but it no longer runs background goroutines
or holds atomic pointers. Long-running callers that want refresh can wire
httpcache.Cacher to atomic.Pointer themselves.
check-ip drops geo.Init/geo.Run — OpenDatabases does the fetch+open work
itself, and a one-shot CLI doesn't need background refresh.
Uses atomic.Pointer[ipcohort.Cohort] directly and builds a per-source
refresh closure (files / git / http). One goroutine drives the ticker.
Exercises what the dataset pkg was abstracting so we can judge which
bits are worth a shared pkg.
Extract the file/git/httpcache mode switch into newSource and the Group
wiring into newBlocklists. main becomes flag parsing + exit code logic
only; run owns ctx and the check. Helpers (loadCohort, cacheDir,
splitCSV, loadWhitelist) are small and single-purpose.
Still exercises dataset.Group + background refresh, gitshallow, and
httpcache as before.
OpenDatabases(confPath, cityPath, asnPath) handles conf discovery, cache
dir setup, and Databases construction. DefaultConfPaths lists the standard
GeoIP.conf locations. cmd/check-ip/geo.go deleted; main calls one function.
Remove View[T] — Add now returns *Dataset[T] directly. Callers use Load()
on the returned Dataset; Init/Run belong to the owning Group.
main.go simplified: declare syncer + file paths per case, then one
g.Init() and one g.Run(). No manual loops over individual datasets.
Add gitshallow.Repo.FilePath helper.
Blocklist:
- Add -inbound, -outbound, -whitelist flags for explicit file paths
- buildSources() replaces the old constructor trio; explicit flags always win
- -data-dir and -git still work as defaults for the bitwire-it layout
GeoIP:
- Auto-discover GeoIP.conf from ./GeoIP.conf then ~/.config/maxmind/GeoIP.conf
- If no conf found and no -city-db/-asn-db given: geoip disabled silently
- If no conf but paths given: use those files (Init fails if absent)
- Each package gets its own go.mod: net/{dataset,httpcache,gitshallow,ipcohort,geoip,formmailer}
- go.work with replace directives for cross-module workspace resolution
- dataset.Syncer/NopSyncer moved here from httpcache; callers duck-type it
- dataset.View[T] returned by Add to prevent Init/Sync/Run misuse on group members
- cmd/check-ip moved from net/ipcohort/cmd/check-ip to top-level cmd/check-ip
- Add net/ipcohort/cmd/ipcohort-contains for standalone cohort membership testing
Replace directives pointing to local paths prevent `go install` from
working. Use published module versions (sqlmigrate v1.0.2, shmigrate
v1.0.2) so users can install with:
go install github.com/therootcompany/golib/cmd/sql-migrate/v2@latest
- Use full flag names in command defaults (--no-align, --tuples-only, etc.)
- Add sqlite/sqlite3/lite alias for SQLite
- Add sqlcmd/mssql/sqlserver alias for SQL Server (go-sqlcmd, TDS 8.0)
- Add SQL Server help text with SQLCMD env vars and TLS docs
- Add sqlite and sqlcmd SELECT expressions for id+name migration log
Migration{ID, Name} is the identity type returned by Up/Down/Latest/Drop.
Script{Migration, Up, Down} holds collected SQL content from Collect().
Migrator interface now takes SQL as a separate parameter:
ExecUp(ctx, Migration, sql string)
ExecDown(ctx, Migration, sql string)
This separates identity from content — callers that track what ran
don't need to carry around SQL strings they'll never use.
Updates shmigrate to match (ignores the sql parameter, references
files on disk instead).
- logMigrationsSelect() returns DB-specific SELECT for id+name output,
matching psql explicitly and erroring on unrecognized --sql-command
- logMigrationsQueryNote const for the comment header
- maybeUpgradeLogQuery() auto-upgrades old _migrations.sql on first run,
replacing only the matching SELECT line
- Add UPGRADING help section pointing to sync subcommand
Passing 0 to Up() or Down() is an easy mistake — it silently means
"all" which could be destructive. Now n=0 returns ErrInvalidN.
Convention: n > 0 for a specific count, n < 0 (typically -1) for all.
Factor the inline migration logic from cmd/sql-migrate into reusable
packages: database/sqlmigrate (core types, matching, file collection)
and database/sqlmigrate/shmigrate (shell script generation backend).
No behavior changes — the CLI produces identical output. The shmigrate
package implements the sqlmigrate.Migrator interface so other backends
(pgmigrate, mymigrate, etc.) can follow the same pattern.
Document how to use PGOPTIONS="-c search_path=..." for multi-tenant
PostgreSQL migrations. Each schema gets its own _migrations table,
so tenants are migrated independently.
An explicit 0 argument should error, not silently run all pending (up)
or get handled as a special case (down). Change the guard from < 0 to
< 1 in both subcommands so '0' is treated as invalid input.
An explicit 0 argument to 'up' should error (like 'down 0' already
does), not silently run all pending migrations. Change the guard
from < 0 to < 1 to match 'down' behavior.
The psql command constant uses -A -t (--no-align --tuples-only) but
the help text and code comments only showed --no-align. Without
--tuples-only, psql prints column headers into migrations.log,
corrupting it.
Replace hardcoded version const with var block populated by goreleaser
ldflags (version, commit, date). Add printVersion() matching the pattern
used by sibling commands (tcpfwd, smsapid, auth-proxy). Fix date var
shadowing in main() by renaming local to today.
The create subcommand generated .up.sql files with INSERT INTO
_migrations as the FIRST statement, before the actual DDL. If the
DDL fails, the migration is incorrectly marked as applied. Move the
INSERT to be the LAST statement, matching how .down.sql already puts
DELETE FROM _migrations last.
Also fix the automatic fixup logic to append (not prepend) missing
INSERT statements to existing .up.sql files.
Fixes#86
Adds `sql-migrate sync` which outputs a shell script that refreshes
the local migrations.log from the DB by running _migrations.sql.
Uses `|| true` so a fresh DB with no _migrations table yields an
empty log (all migrations pending) rather than an error.
Usage:
sql-migrate -d ./db/migrations sync | sh
sql-migrate -d ./db/migrations up | sh
