Unrecognized non-empty values keep the trimmed original in Raw() so
callers can apply column-specific word policy post-parse (e.g. reading
"disabled" as true in a Disabled column) instead of string-matching the
error message.
BoolishTimestamp defaulted unrecognized non-empty values to true: any
word it does not understand (a misspelling, "disabled", "banana")
behaved as on. Unrecognized values are now valid+false with the error
retained so callers can apply their own policy, and "disabled"/"disable"
are no longer true-like.
Both bool types now read the human "not supplied" set through one shared
isBoolNull sentinel: empty/whitespace-only, the exact PG dump sentinel
\N (case-sensitive), and the case-insensitive words null, \null, none,
unknown. Bool previously reached absent only by ParseBool failing; it
now matches BoolishTimestamp explicitly. Free-text types must not reuse
this set: most of these words are legitimate values there.
Human-maintained sources (e.g. Google Sheets exports) routinely carry
stray blank rows. A blank row above the header was taken as the header,
so every data row decoded to zero values; blank data rows decoded as
empty records.
ParseCSV now wraps the stream in a streaming csvutil.Reader (skipBlanks)
that drops records whose cells are all empty or whitespace-only. Records
decode one at a time with a single record buffered; delimiter and
ragged-row behavior are unchanged, and all-blank input yields nil, nil.
Some deployments used "source,comment" as the CSV header for IP list
files, but the parser only recognized "network" as a header row.
Accept both "network" and "source" (case-insensitive) to avoid
treating the header as a (non-existent) domain entry.
Rename net/ipgate to net/ippolicy and compose independently refreshed
whitelist and blacklist sources into one IP decision policy.
- rename slog group and error prefixes from ipgate to ippolicy
- delete unused ParseDomainSet and RecordReader from domainsetcsv.go
- rewrite README to match the actual Config API
- document nil-whitelist behavior on New()
- bump iplist dependency to v0.8.5
- go mod tidy: remove stale ipgate and iplist entries
Add net/iplist for shared IP-policy source loading.
- Accept local TSV/CSV file paths or HTTP(S) URLs (including Google Sheets).
- Auto-detect TSV vs CSV format from file content.
- Send Accept: text/tab-separated-values header for content negotiation.
- Parse the first column and ignore comments (#) and optional "network" header.
- Expand nested list URLs up to MaxNestedSources (2) levels.
- Support Basic Auth from URL userinfo without sending credentials in the URL.
- Persist cached URL responses across restarts via httpcache.
- Refresh periodically and preserve the known-good cache on fetch or parse failure.
- Validate entries as IP address, CIDR range, or domain name (Validate()).
- Use golib/https.NewInternalClient() through httpcache.NewWith.
- Add README with usage examples and entry format documentation.
Add non-breaking context-aware and custom-HTTP-client variants of the
existing reader and fetch functions:
- NewReaderFromContext / NewReaderFromContextWith
- NewReaderFromURLContext / NewReaderFromURLWith
- NewReaderFromIDsContext
- GetSheetContext
All existing function signatures are unchanged. The shared response
handling is extracted into newReaderFromResponse, and the legacy GetSheet
path is refactored through getSheetWithGet for clarity.
Ensure cohort is never nil:
- NewDomainSet stores an empty Cohort before rebuildCohort, so a parse
failure leaves an empty (non-nil) cohort instead of nil
- Contains uses CompareAndSwap to lazily self-initialise a zero-value
struct, making nil impossible after first access — no explicit nil
check needed
Extract the cache-validity check into a testable cachedCohortValid()
helper and add filesPresent() to detect when data files exist on disk.
Use strings.Builder in commaify() for efficiency.
Tests:
- TestPrefixSetContainsZeroValue: zero-value self-heals via CAS,
cohort is non-nil after first Contains call
- TestCachedCohortValidRequiresFiles: cache invalidation logic
