feat(skills): add sqlmigrate skill index and per-database skills

Index skill (use-sqlmigrate) plus focused skills for CLI usage, Go
library integration, and per-database conventions (PostgreSQL,
MySQL/MariaDB, SQLite, SQL Server).
This commit is contained in:
AJ ONeal 2026-04-09 14:23:40 -06:00
parent 5b3a4be3c2
commit 0c1eb1f125
No known key found for this signature in database
7 changed files with 609 additions and 0 deletions

View File

@ -0,0 +1,117 @@
---
name: use-sql-migrate-cli
description: sql-migrate CLI tool for database migrations. Use when initializing migrations, creating migration files, running up/down, checking status, or generating migration scripts. Covers psql, mariadb, mysql, sqlite, sqlcmd.
depends: [use-sqlmigrate]
---
## Install
```sh
webi go
go install github.com/therootcompany/golib/cmd/sql-migrate/v2@latest
```
## Commands
### init
```sh
sql-migrate -d ./sql/migrations/ init --sql-command <psql|mariadb|mysql|sqlite|sqlcmd>
```
Creates: migrations directory, `0001-01-01-001000_init-migrations.{up,down}.sql`,
`migrations.log`, `_migrations.sql` query file.
MUST: Run the generated init script to create the `_migrations` table:
```sh
sql-migrate -d ./sql/migrations/ up | sh
```
### create
```sh
sql-migrate -d ./sql/migrations/ create add-user-tables
```
Generates a canonically-named up/down pair with a random 8-hex-char ID:
```
2026-04-09-001000_add-user-tables.up.sql
2026-04-09-001000_add-user-tables.down.sql
```
If files for today already exist, the number increments by 1000.
### up / down
```sh
# apply ALL pending migrations
sql-migrate -d ./sql/migrations/ up | sh
# apply next 2 pending
sql-migrate -d ./sql/migrations/ up 2 | sh
# roll back 1 (default)
sql-migrate -d ./sql/migrations/ down | sh
# roll back 3
sql-migrate -d ./sql/migrations/ down 3 | sh
```
Output is a shell script. Review before piping to `sh`.
### status
```sh
sql-migrate -d ./sql/migrations/ status
```
Shows applied (reverse order) and pending migrations. Does not execute anything.
### sync
```sh
sql-migrate -d ./sql/migrations/ sync | sh
```
Reloads `migrations.log` from the database. Run after upgrading sql-migrate.
### list
```sh
sql-migrate -d ./sql/migrations/ list
```
Lists all up/down migration files found.
## Options
| Flag | Default | Purpose |
|------|---------|---------|
| `-d <dir>` | `./sql/migrations/` | Migrations directory |
| `--sql-command` | `psql` | SQL command template (init only) |
| `--migrations-log` | `../migrations.log` | Log file path relative to migrations dir (init only) |
## SQL command aliases
| Alias | Expands to |
|-------|-----------|
| `psql`, `postgres`, `postgresql`, `pg`, `plpgsql` | `psql "$PG_URL" -v ON_ERROR_STOP=on --no-align --tuples-only --file %s` |
| `mariadb` | `mariadb --defaults-extra-file="$MY_CNF" --silent --skip-column-names --raw < %s` |
| `mysql`, `my` | `mysql --defaults-extra-file="$MY_CNF" --silent --skip-column-names --raw < %s` |
| `sqlite`, `sqlite3`, `lite` | `sqlite3 "$SQLITE_PATH" < %s` |
| `sqlcmd`, `mssql`, `sqlserver` | `sqlcmd --exit-on-error --headers -1 --trim-spaces --encrypt-connection strict --input-file %s` |
Custom commands: pass any string with `%s` as the file placeholder.
## Configuration
Stored in the initial migration file as comments:
```sql
-- sql_command: psql "$PG_URL" -v ON_ERROR_STOP=on --no-align --tuples-only --file %s
-- migrations_log: ../migrations.log
```
These are read by the CLI on every run. Edit them to change the sql command or log path.

View File

@ -0,0 +1,151 @@
---
name: use-sql-migrate-golang
description: Embed SQL migrations in Go applications using sqlmigrate library. Use when writing Go code that runs migrations on startup, implements auto-migrate, or uses the Migrator interface. Covers pgmigrate, mymigrate, litemigrate, msmigrate.
depends: [use-sqlmigrate, go-stack]
---
## Modules
Each backend is a separate Go module. Import only what you need:
| Module | Import path |
|--------|-------------|
| Core | `github.com/therootcompany/golib/database/sqlmigrate` |
| PostgreSQL | `github.com/therootcompany/golib/database/sqlmigrate/pgmigrate` |
| MySQL/MariaDB | `github.com/therootcompany/golib/database/sqlmigrate/mymigrate` |
| SQLite | `github.com/therootcompany/golib/database/sqlmigrate/litemigrate` |
| SQL Server | `github.com/therootcompany/golib/database/sqlmigrate/msmigrate` |
## Core API
```go
// Collect reads .up.sql/.down.sql pairs from an fs.FS
scripts, err := sqlmigrate.Collect(migrationsFS, "sql/migrations")
// Apply all pending migrations
applied, err := sqlmigrate.Latest(ctx, runner, scripts)
// Apply n pending migrations (-1 = all)
applied, err := sqlmigrate.Up(ctx, runner, scripts, n)
// Roll back n migrations (-1 = all, default pattern: 1)
rolled, err := sqlmigrate.Down(ctx, runner, scripts, n)
// Roll back all migrations
rolled, err := sqlmigrate.Drop(ctx, runner, scripts)
// Check status
status, err := sqlmigrate.GetStatus(ctx, runner, scripts)
// status.Applied, status.Pending
```
## Key types
```go
type Migration struct {
ID string // 8-char hex from INSERT statement
Name string // e.g. "2026-04-05-001000_create-todos"
}
type Script struct {
Migration
Up string // .up.sql content
Down string // .down.sql content
}
type Migrator interface {
ExecUp(ctx context.Context, m Migration, sql string) error
ExecDown(ctx context.Context, m Migration, sql string) error
Applied(ctx context.Context) ([]Migration, error)
}
```
## Embedding migrations
MUST: Use `embed.FS` to bundle migration files into the binary:
```go
//go:embed sql/migrations/*.sql
var migrationsFS embed.FS
```
## Backend setup pattern
MUST: Backends take a single connection, not a pool.
### database/sql backends (MySQL, SQLite, SQL Server)
```go
db, err := sql.Open("mysql", dsn)
// ...
// acquire a dedicated connection for migrations
conn, err := db.Conn(ctx)
// ...
defer func() { _ = conn.Close() }()
runner := mymigrate.New(conn) // or litemigrate.New(conn), msmigrate.New(conn)
```
### pgx backend (PostgreSQL)
```go
// single connection, not pool
conn, err := pgx.Connect(ctx, pgURL)
// ...
defer func() { _ = conn.Close(ctx) }()
runner := pgmigrate.New(conn)
```
## Auto-migrate on startup
Common pattern — run all pending migrations before serving:
```go
func main() {
// ... open db, get conn ...
scripts := mustCollectMigrations()
runner := litemigrate.New(conn)
// apply all pending (idempotent)
if _, err := sqlmigrate.Latest(ctx, runner, scripts); err != nil {
log.Fatalf("auto-migrate: %v", err)
}
// close migration conn, use db/pool for app queries
_ = conn.Close()
// ... start serving ...
}
```
## Example app structure
```
my-app/
main.go # flag parsing, DB setup, auto-migrate, dispatch
demo.go # app-specific CRUD (uses *sql.DB for queries)
go.mod
sql/
migrations/
0001-01-01-001000_init-migrations.up.sql
0001-01-01-001000_init-migrations.down.sql
2026-04-05-001000_create-todos.up.sql
2026-04-05-001000_create-todos.down.sql
```
## Migrate subcommand pattern
Expose `migrate up/down/status/reset` as a subcommand:
```go
case "migrate":
err = runMigrate(ctx, runner, migrations, subArgs)
case "add":
autoMigrate(ctx, runner, migrations)
err = runAdd(ctx, db, subArgs)
```
See example apps in `cmd/sql-migrate/examples/` for full implementations.

View File

@ -0,0 +1,67 @@
---
name: use-sql-migrate-mysql
description: MySQL and MariaDB migrations with sql-migrate and mymigrate. Use when setting up MySQL/MariaDB migrations, configuring multiStatements, or MY_CNF credentials.
depends: [use-sqlmigrate]
---
## CLI setup
```sh
# MariaDB
sql-migrate -d ./sql/migrations/ init --sql-command mariadb
# MySQL
sql-migrate -d ./sql/migrations/ init --sql-command mysql
```
## Environment
```sh
# .env
MY_URL='user:pass@tcp(localhost:3306)/mydb?multiStatements=true&parseTime=true'
MY_CNF='./my.cnf'
```
MUST: Include `multiStatements=true` in the DSN. mymigrate validates this on first exec and returns an error if missing.
## Credentials file (for CLI)
The CLI uses `--defaults-extra-file` to avoid passwords in command args:
```ini
# my.cnf
[client]
host=localhost
port=3306
database=mydb
user=appuser
password=secret
```
## Go library
```go
import (
"database/sql"
_ "github.com/go-sql-driver/mysql"
"github.com/therootcompany/golib/database/sqlmigrate"
"github.com/therootcompany/golib/database/sqlmigrate/mymigrate"
)
db, err := sql.Open("mysql", myURL)
conn, err := db.Conn(ctx)
defer func() { _ = conn.Close() }()
runner := mymigrate.New(conn)
applied, err := sqlmigrate.Latest(ctx, runner, scripts)
```
## SQL dialect notes
- DDL statements (CREATE/ALTER/DROP) auto-commit in MySQL — partial failures possible on multi-statement down migrations
- `INSERT IGNORE` for idempotent seeds (not `ON CONFLICT`)
- `NOW()` for current timestamp
- String concatenation: `CONCAT(id, CHAR(9), name)` (used by sync query)
- `ON UPDATE CURRENT_TIMESTAMP` for auto-updated timestamps
- Error 1146 = table doesn't exist (handled automatically by mymigrate)

View File

@ -0,0 +1,64 @@
---
name: use-sql-migrate-postgres
description: PostgreSQL migrations with sql-migrate and pgmigrate. Use when setting up PostgreSQL migrations, schema multi-tenancy, or pgx connection for migrations.
depends: [use-sqlmigrate]
---
## CLI setup
```sh
sql-migrate -d ./sql/migrations/ init --sql-command psql
```
## Environment
```sh
# .env
PG_URL='postgres://user:pass@localhost:5432/mydb?sslmode=disable'
```
## Go library
```go
import (
"github.com/jackc/pgx/v5"
"github.com/therootcompany/golib/database/sqlmigrate"
"github.com/therootcompany/golib/database/sqlmigrate/pgmigrate"
)
// MUST: use pgx.Connect (single conn), not pgxpool.New
conn, err := pgx.Connect(ctx, pgURL)
defer func() { _ = conn.Close(ctx) }()
runner := pgmigrate.New(conn)
applied, err := sqlmigrate.Latest(ctx, runner, scripts)
```
## Schema multi-tenancy
Each PostgreSQL schema gets its own `_migrations` table. Tenants are migrated independently.
### CLI
```sh
PGOPTIONS="-c search_path=tenant123" sql-migrate -d ./sql/migrations/ up | sh
```
### Go library
```go
conn, err := pgx.Connect(ctx, pgURL)
_, err = conn.Exec(ctx, fmt.Sprintf(
"SET search_path TO %s",
pgx.Identifier{schema}.Sanitize(),
))
runner := pgmigrate.New(conn)
```
## SQL dialect notes
- `CREATE TABLE IF NOT EXISTS` works
- `ON CONFLICT DO NOTHING` for idempotent seeds
- String concatenation: `id || CHR(9) || name` (used by sync query)
- Timestamps: `TIMESTAMP DEFAULT CURRENT_TIMESTAMP`
- Error code 42P01 = table doesn't exist (handled automatically by pgmigrate)

View File

@ -0,0 +1,70 @@
---
name: use-sql-migrate-sqlite
description: SQLite migrations with sql-migrate and litemigrate. Use when setting up SQLite migrations, configuring foreign keys, or using modernc.org/sqlite driver.
depends: [use-sqlmigrate]
---
## CLI setup
```sh
sql-migrate -d ./sql/migrations/ init --sql-command sqlite
```
## Environment
```sh
# .env
SQLITE_PATH='./app.db'
```
## Go library
```go
import (
"database/sql"
_ "modernc.org/sqlite"
"github.com/therootcompany/golib/database/sqlmigrate"
"github.com/therootcompany/golib/database/sqlmigrate/litemigrate"
)
// MUST: enable foreign keys via pragma
db, err := sql.Open("sqlite", dbPath+"?_pragma=foreign_keys(1)")
conn, err := db.Conn(ctx)
defer func() { _ = conn.Close() }()
runner := litemigrate.New(conn)
applied, err := sqlmigrate.Latest(ctx, runner, scripts)
```
MUST: The caller imports the SQLite driver (`modernc.org/sqlite` recommended — pure Go, no CGo).
## SQL dialect notes
- `datetime('now')` instead of `CURRENT_TIMESTAMP` for default values in expressions
- `TEXT` for timestamp columns (SQLite has no native datetime type)
- `INSERT OR IGNORE` for idempotent seeds (not `INSERT IGNORE`)
- String concatenation: `id || CHAR(9) || name` (used by sync query)
- `ALTER TABLE ... DROP COLUMN` requires SQLite 3.35.0+ (2021-03-12)
- "no such table" error string used to detect missing `_migrations` table
- Default path: `todos.db` if no env var set
## sqlc with SQLite
SQLite `CHAR(n)` columns map to `interface{}` in sqlc. Use column-level overrides:
```yaml
# sqlc.yaml
sql:
- schema: "sql/migrations/"
queries: "sql/queries/"
engine: "sqlite"
gen:
go:
out: "internal/tododb"
overrides:
- column: "todos.id"
go_type: "string"
- column: "todos.status"
go_type: "string"
```

View File

@ -0,0 +1,95 @@
---
name: use-sql-migrate-sqlserver
description: SQL Server migrations with sql-migrate and msmigrate. Use when setting up SQL Server migrations, configuring sqlcmd, TDS 8.0 encryption, or SQLCMD environment variables.
depends: [use-sqlmigrate]
---
## CLI setup
```sh
sql-migrate -d ./sql/migrations/ init --sql-command sqlcmd
```
Requires the modern sqlcmd (go-mssqldb), not the legacy ODBC version.
Install: `brew install sqlcmd` (macOS), `winget install sqlcmd` (Windows).
## Environment
```sh
# .env — for Go app (DSN)
MS_URL='sqlserver://sa:Password@localhost:1433?database=mydb'
# .env — for sqlcmd CLI (reads these automatically)
SQLCMDSERVER='localhost'
SQLCMDDATABASE='mydb'
SQLCMDUSER='sa'
SQLCMDPASSWORD='secret'
```
SQLCMDSERVER formats:
- `localhost` — default instance
- `'localhost\SQLEXPRESS'` — named instance (quote the backslash)
- `'localhost,1433'` — host and port
## TDS 8.0 encryption
Default uses `--encrypt-connection strict` (TLS-first with ALPN `tds/8.0` and SNI).
For local dev without TLS:
```sh
sql-migrate -d ./sql/migrations/ init \
--sql-command 'sqlcmd --exit-on-error --headers -1 --trim-spaces --encrypt-connection disable --input-file %s'
```
## Go library
```go
import (
"database/sql"
_ "github.com/microsoft/go-mssqldb"
"github.com/therootcompany/golib/database/sqlmigrate"
"github.com/therootcompany/golib/database/sqlmigrate/msmigrate"
)
db, err := sql.Open("sqlserver", msURL)
conn, err := db.Conn(ctx)
defer func() { _ = conn.Close() }()
runner := msmigrate.New(conn)
applied, err := sqlmigrate.Latest(ctx, runner, scripts)
```
## SQL dialect notes
- `IF OBJECT_ID('table', 'U') IS NULL CREATE TABLE ...` instead of `CREATE TABLE IF NOT EXISTS`
- `SYSDATETIME()` instead of `CURRENT_TIMESTAMP` for DATETIME2 defaults
- `DATETIME2` for timestamps (not `TIMESTAMP` — that's a row version in SQL Server)
- `@p1`, `@p2` for parameterized queries in Go (not `?`)
- Dropping columns with defaults requires dropping the default constraint first:
```sql
DECLARE @constraint NVARCHAR(256);
SELECT @constraint = name FROM sys.default_constraints
WHERE parent_object_id = OBJECT_ID('todos')
AND parent_column_id = (SELECT column_id FROM sys.columns
WHERE object_id = OBJECT_ID('todos') AND name = 'priority');
IF @constraint IS NOT NULL
EXEC('ALTER TABLE todos DROP CONSTRAINT ' + @constraint);
ALTER TABLE todos DROP COLUMN priority;
```
- `IF NOT EXISTS (SELECT 1 FROM table WHERE ...) INSERT ...` for idempotent seeds
- String concatenation: `id + CHAR(9) + name` (used by sync query)
- Error 208 = invalid object name (table doesn't exist, handled by msmigrate)
## SSH tunnel for remote dev
```sh
ssh -o ProxyCommand='sclient --alpn ssh %h' -fnNT \
-L 21433:localhost:1433 \
tls-<ip>.a.bnna.net
```
Then set `MS_URL='sqlserver://sa:pass@localhost:21433?database=todos'`.

View File

@ -0,0 +1,45 @@
---
name: use-sqlmigrate
description: Database migration tools for Go projects. Use when writing migrations, running sql-migrate CLI, embedding migrations in Go apps, or setting up new database schemas. Covers PostgreSQL, MySQL/MariaDB, SQLite, SQL Server.
depends: [go-stack]
---
## Overview
sqlmigrate is a feature-branch-friendly SQL migration system with two modes:
1. **CLI** (`sql-migrate`) — generates shell scripts that pipe to `sh`
2. **Go library** (`sqlmigrate` + backend) — embed migrations in Go binaries
Both use the same migration file format and `_migrations` tracking table.
## Focused skills
| Skill | When to use |
|-------|-------------|
| `use-sql-migrate-cli` | CLI tool: init, create, up, down, sync, status |
| `use-sql-migrate-golang` | Go library: embed migrations, Migrator interface, auto-migrate on startup |
| `use-sql-migrate-postgres` | PostgreSQL: pgx connection, schema multi-tenancy, PGOPTIONS |
| `use-sql-migrate-mysql` | MySQL/MariaDB: multiStatements DSN, MY_CNF, mariadb vs mysql |
| `use-sql-migrate-sqlite` | SQLite: foreign keys pragma, modernc.org/sqlite driver |
| `use-sql-migrate-sqlserver` | SQL Server: sqlcmd, TDS 8.0 encryption, SQLCMD* env vars |
## Migration file format
```
<yyyy-mm-dd>-<number>_<name>.<up|down>.sql
2026-04-05-001000_create-todos.up.sql
2026-04-05-001000_create-todos.down.sql
```
- Numbers increment by 1000 (allows inserting between)
- Initial migration: `0001-01-01-001000_init-migrations`
- Each `.up.sql` MUST end with `INSERT INTO _migrations (name, id) VALUES ('<name>', '<8-hex-id>');`
- Each `.down.sql` MUST end with `DELETE FROM _migrations WHERE id = '<8-hex-id>';`
## Key design decisions
- **Feature-branch friendly**: no sequential numbering, no conflicts
- **ID-based matching**: migrations matched by 8-char hex ID, not name — safe to rename
- **Shell-first CLI**: generates reviewable scripts, never executes directly
- **Separate Go modules**: each backend is its own module to avoid pulling unnecessary drivers