Compare commits

...

10 commits

Author SHA1 Message Date
459ff47ed9 Add deprecated DNS record types checker
Some checks reported errors
continuous-integration/drone/push Build was killed
Detects usage of obsolete record types (SPF, A6, NXT, SIG, KEY, WKS) in
services and reports them with the relevant RFC deprecation reason.
2026-04-23 16:32:27 +07:00
7a504eabe3 Add NS security restrictions checker
Checks nameservers for common security misconfigurations: AXFR/IXFR zone
transfer acceptance, recursion availability, RFC 8482 ANY query handling,
and authoritative status.
2026-04-23 16:32:27 +07:00
3695de0959 Add TLS checker
Some checks failed
continuous-integration/drone/push Build is failing
2026-04-23 16:27:44 +07:00
86e481cd10 checkers: migrate to checker-sdk-go v1.2.0 []CheckState signature
Rules now return []CheckState, the engine stamps RuleName from the rule,
and the HTTP rule-result lookup matches on RuleName rather than Code.
domain_contact emits one state per role (Subject) instead of a
concatenated single-state message.
2026-04-23 16:24:25 +07:00
08ea0a523f backup: include cross-checker discovery entries in backup/restore
Extend Backup to carry the two new KV indexes introduced by the
discovery mechanism.
2026-04-22 19:44:38 +07:00
e52af671a0 checkers: resolve Related in ReportContext for HTML + metrics reports
Complete the ReportContext composition path so reporters can fold
downstream observations into their output:

  - checker.BuildReportContext wraps a raw payload plus the engine's
    RelatedObservationLookup in a lazy ReportContext: Related(key) is
    resolved on first access and cached. When no lookup is wired the
    context falls back to sdk.StaticReportContext, matching the
    pre-existing behaviour.
  - GetHTMLReportWithContext / GetMetricsWithContext: new helpers that
    accept a pre-built ReportContext, for callers that want to feed
    Related into a reporter explicitly.
  - The execution controller now builds a ReportContext via the
    engine's RelatedLookup method before calling the HTML reporter.
    When the engine is wired with discovery storage, the reporter sees
    the producer's consumer lineage through ctx.Related(consumerKey).
  - HTTPObservationProvider implements CheckerHTMLReporter and
    CheckerMetricsReporter: both forward to POST /report with
    ExternalReportRequest{Key, Data, Related}. A 501 response is
    surfaced as an explicit "does not support /report" error. These
    methods are available for callers that want to route reports to
    remote checkers; the default in-process reporter dispatch is
    unchanged.
2026-04-22 19:44:38 +07:00
ff2b8a21f2 checkers: compose cross-checker observations via GetRelated
Close the discovery loop described in docs/checker-discovery.md: entries
published in commit 3 now feed consumer checkers, and their observations
flow back to the original producer.

Three tightly-coupled changes:

  - CheckerOptionsUsecase gains an optional DiscoveryEntryStorage
    dependency (WithDiscoveryEntryStore). When a checker declares
    AutoFill="discovery_entries" on an option,
    BuildMergedCheckerOptionsWithAutoFill populates it with the entries
    stored for the target: all producers, no host-side filtering by
    Type. The method also returns the concrete list of entries injected
    so the engine can persist lineage for them.

  - CheckerEngine records a DiscoveryObservationRef per (entry, obs key)
    tuple after the snapshot is stored. The ref namespaces back to the
    *producer* (ProducerID, Target, Ref) while carrying the consumer's
    key and the snapshot pointer, so a later GetRelated from the
    producer can reach the consumer's observation in one lookup.

  - ObservationContext exposes SetRelatedLookup (called once per run by
    the engine) and implements GetRelated on top of the installed
    closure. The engine's closure walks the producer's published
    entries, resolves each ref's observation refs, loads the snapshots,
    and materialises []RelatedObservation. Stale refs (entry gone,
    snapshot TTL'd) are skipped silently: implicit GC, as the doc
    permits.
2026-04-22 19:44:38 +07:00
5cc1135a67 checkers: harvest discovery entries during collection
Wire the newly-added DiscoveryEntryStorage into the execution pipeline:

  - ObservationContext tracks DiscoveryEntry records published by each
    provider. After Collect, providers that implement DiscoveryPublisher
    are asked for their entries (on the native Go value, no JSON round
    trip), and the results are cached by observation key.
  - HTTPObservationProvider also implements DiscoveryPublisher: it
    records the Entries field of the remote /collect response and
    surfaces them through DiscoverEntries. Each override instance is
    scoped to a single execution run, so no locking is needed.
  - CheckerEngine.runPipeline calls ReplaceDiscoveryEntries after
    persisting the snapshot, always replacing the previous set for
    (checkerID, target), including when a run produces none, so stale
    entries from earlier cycles self-heal.
2026-04-22 19:44:38 +07:00
c069006951 checkers: add storage for discovery entries and observation lineage
Introduce the two KV indexes that back the cross-checker discovery
mechanism described in docs/checker-discovery.md:

  dscent|{producer}|{target}|{type}|{ref}         primary record
  dscent-tgt|{target}|{producer}|{type}|{ref}     target lookup (auto-fill)
  dscobs|{producer}|{target}|{ref}|{consumer}|{k} observation lineage
  dscobs-snap|{snapshotId}|...                     cascade on snapshot delete

ReplaceDiscoveryEntries is the canonical publication path: the whole
set previously stored for (producer, target) is cleared, then the new
set is written. The observation-lineage side uses a single upsert per
(producer, target, ref, consumer, key) tuple, with a snapshot-scoped
reverse index so deleting a snapshot cascades cleanly. Putting a ref
under a new snapshot removes the previous snap-index so a later
cascade on the old snapshot does not wipe the refreshed primary.

Adds StoredDiscoveryEntry and DiscoveryObservationRef to the host-only
model, DiscoveryEntryStorage / DiscoveryObservationStorage to the
checker usecase storage surface, embeds both in storage.Storage, and
regenerates the instrumented wrapper. Unit tests cover round-trip,
atomic replace, multi-producer aggregation, upsert, and cascade
delete.

No pipeline wiring yet.
2026-04-22 19:44:38 +07:00
60e996065f checkers: adopt ReportContext-based reporter signatures
Update happyDomain to the new checker-sdk-go reporter contract, where
CheckerHTMLReporter.GetHTMLReport and CheckerMetricsReporter.ExtractMetrics
take a ReportContext instead of a raw json.RawMessage. The ReportContext
will later carry cross-checker related observations; for now every call
site wraps the raw payload via sdk.StaticReportContext, so behavior is
unchanged.

Also re-export the new discovery-related SDK types (DiscoveryEntry,
DiscoveryPublisher, RelatedObservation, ReportContext,
AutoFillDiscoveryEntries) as aliases under happydns, and satisfy the
extended ObservationGetter interface on ObservationContext and the
test stub with a no-op GetRelated.

No new behavior: plumbing for the upcoming discovery pipeline.
2026-04-22 19:44:38 +07:00
33 changed files with 2234 additions and 175 deletions

View file

@ -92,14 +92,14 @@ func (r *domainContactRule) ValidateOptions(opts happydns.CheckerOptions) error
return nil
}
func (r *domainContactRule) Evaluate(ctx context.Context, obs happydns.ObservationGetter, opts happydns.CheckerOptions) happydns.CheckState {
func (r *domainContactRule) Evaluate(ctx context.Context, obs happydns.ObservationGetter, opts happydns.CheckerOptions) []happydns.CheckState {
var whois WHOISData
if err := obs.Get(ctx, ObservationKeyWhois, &whois); err != nil {
return happydns.CheckState{
return []happydns.CheckState{{
Status: happydns.StatusError,
Message: fmt.Sprintf("Failed to get WHOIS data: %v", err),
Code: "contact_error",
}
}}
}
expectedName, _ := opts["expectedName"].(string)
@ -107,11 +107,11 @@ func (r *domainContactRule) Evaluate(ctx context.Context, obs happydns.Observati
expectedEmail, _ := opts["expectedEmail"].(string)
if expectedName == "" && expectedOrg == "" && expectedEmail == "" {
return happydns.CheckState{
return []happydns.CheckState{{
Status: happydns.StatusUnknown,
Message: "No expected contact values configured",
Code: "contact_skipped",
}
}}
}
checkRolesStr := "registrant"
@ -127,27 +127,33 @@ func (r *domainContactRule) Evaluate(ctx context.Context, obs happydns.Observati
}
}
if len(roles) == 0 {
return happydns.CheckState{
return []happydns.CheckState{{
Status: happydns.StatusUnknown,
Message: "No contact roles to check",
Code: "contact_skipped",
}
}}
}
worst := happydns.StatusOK
var lines []string
out := make([]happydns.CheckState, 0, len(roles))
for _, role := range roles {
contact, found := whois.Contacts[role]
if !found || contact == nil {
lines = append(lines, fmt.Sprintf("%s: contact not found", role))
worst = worseStatus(worst, happydns.StatusWarn)
out = append(out, happydns.CheckState{
Status: happydns.StatusWarn,
Message: "contact not found",
Code: "contact_missing",
Subject: role,
})
continue
}
if isRedacted(contact) {
lines = append(lines, fmt.Sprintf("%s: contact info is redacted/private", role))
worst = worseStatus(worst, happydns.StatusInfo)
out = append(out, happydns.CheckState{
Status: happydns.StatusInfo,
Message: "contact info is redacted/private",
Code: "contact_redacted",
Subject: role,
})
continue
}
@ -163,18 +169,23 @@ func (r *domainContactRule) Evaluate(ctx context.Context, obs happydns.Observati
}
if len(mismatches) > 0 {
lines = append(lines, fmt.Sprintf("%s: %s", role, strings.Join(mismatches, ", ")))
worst = worseStatus(worst, happydns.StatusWarn)
out = append(out, happydns.CheckState{
Status: happydns.StatusWarn,
Message: strings.Join(mismatches, ", "),
Code: "contact_mismatch",
Subject: role,
})
} else {
lines = append(lines, fmt.Sprintf("%s: contact info matches", role))
out = append(out, happydns.CheckState{
Status: happydns.StatusOK,
Message: "contact info matches",
Code: "contact_ok",
Subject: role,
})
}
}
return happydns.CheckState{
Status: worst,
Message: strings.Join(lines, "; "),
Code: "contact_result",
}
return out
}
// isRedacted reports whether a contact's fields look privacy-protected.

View file

@ -54,16 +54,19 @@ func TestDomainContactRule_Evaluate(t *testing.T) {
obs := newWhoisObs(&WHOISData{Contacts: contactsFixture()})
cases := []struct {
name string
opts happydns.CheckerOptions
want happydns.Status
code string
name string
opts happydns.CheckerOptions
wantWorst happydns.Status
// wantCodes: if non-nil, expect one state per entry with the listed code
// (order matches roles). If nil, expect a single state and use wantCode.
wantCodes []string
wantCode string
}{
{
name: "no expectations",
opts: nil,
want: happydns.StatusUnknown,
code: "contact_skipped",
name: "no expectations",
opts: nil,
wantWorst: happydns.StatusUnknown,
wantCode: "contact_skipped",
},
{
name: "registrant matches",
@ -71,16 +74,16 @@ func TestDomainContactRule_Evaluate(t *testing.T) {
"expectedName": "Alice Example",
"expectedEmail": "alice@example.com",
},
want: happydns.StatusOK,
code: "contact_result",
wantWorst: happydns.StatusOK,
wantCodes: []string{"contact_ok"},
},
{
name: "registrant name mismatch",
opts: happydns.CheckerOptions{
"expectedName": "Carol Other",
},
want: happydns.StatusWarn,
code: "contact_result",
wantWorst: happydns.StatusWarn,
wantCodes: []string{"contact_mismatch"},
},
{
name: "admin role is redacted",
@ -88,8 +91,8 @@ func TestDomainContactRule_Evaluate(t *testing.T) {
"checkRoles": "admin",
"expectedName": "Alice Example",
},
want: happydns.StatusInfo,
code: "contact_result",
wantWorst: happydns.StatusInfo,
wantCodes: []string{"contact_redacted"},
},
{
name: "missing role",
@ -97,8 +100,8 @@ func TestDomainContactRule_Evaluate(t *testing.T) {
"checkRoles": "billing",
"expectedName": "Alice Example",
},
want: happydns.StatusWarn,
code: "contact_result",
wantWorst: happydns.StatusWarn,
wantCodes: []string{"contact_missing"},
},
{
name: "multi-role mixed (worst wins)",
@ -106,20 +109,43 @@ func TestDomainContactRule_Evaluate(t *testing.T) {
"checkRoles": "registrant,admin",
"expectedName": "Alice Example",
},
// admin is redacted (Info) — Info is worse than OK from registrant.
want: happydns.StatusInfo,
code: "contact_result",
// registrant matches (OK), admin is redacted (Info). Info is worst.
wantWorst: happydns.StatusInfo,
wantCodes: []string{"contact_ok", "contact_redacted"},
},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
st := rule.Evaluate(context.Background(), obs, tc.opts)
if st.Status != tc.want {
t.Errorf("status = %v, want %v (msg=%q)", st.Status, tc.want, st.Message)
states := rule.Evaluate(context.Background(), obs, tc.opts)
if tc.wantCodes == nil {
if len(states) != 1 {
t.Fatalf("expected 1 state, got %d", len(states))
}
st := states[0]
if st.Status != tc.wantWorst {
t.Errorf("status = %v, want %v (msg=%q)", st.Status, tc.wantWorst, st.Message)
}
if st.Code != tc.wantCode {
t.Errorf("code = %q, want %q", st.Code, tc.wantCode)
}
return
}
if st.Code != tc.code {
t.Errorf("code = %q, want %q", st.Code, tc.code)
if len(states) != len(tc.wantCodes) {
t.Fatalf("state count = %d, want %d", len(states), len(tc.wantCodes))
}
worst := happydns.StatusOK
for i, st := range states {
if st.Code != tc.wantCodes[i] {
t.Errorf("state[%d].code = %q, want %q", i, st.Code, tc.wantCodes[i])
}
if st.Subject == "" {
t.Errorf("state[%d].Subject is empty", i)
}
worst = worseStatus(worst, st.Status)
}
if worst != tc.wantWorst {
t.Errorf("worst status = %v, want %v", worst, tc.wantWorst)
}
})
}
@ -128,7 +154,11 @@ func TestDomainContactRule_Evaluate(t *testing.T) {
func TestDomainContactRule_EvaluateObservationError(t *testing.T) {
rule := &domainContactRule{}
obs := &stubObservationGetter{key: ObservationKeyWhois, err: errString("nope")}
st := rule.Evaluate(context.Background(), obs, happydns.CheckerOptions{"expectedName": "x"})
states := rule.Evaluate(context.Background(), obs, happydns.CheckerOptions{"expectedName": "x"})
if len(states) != 1 {
t.Fatalf("expected 1 state, got %d", len(states))
}
st := states[0]
if st.Status != happydns.StatusError || st.Code != "contact_error" {
t.Errorf("got %v / %q", st.Status, st.Code)
}
@ -199,9 +229,13 @@ func TestWorseStatus(t *testing.T) {
func TestDomainContactRule_NilContacts(t *testing.T) {
rule := &domainContactRule{}
obs := newWhoisObs(&WHOISData{})
st := rule.Evaluate(context.Background(), obs, happydns.CheckerOptions{
states := rule.Evaluate(context.Background(), obs, happydns.CheckerOptions{
"expectedName": "Alice",
})
if len(states) != 1 {
t.Fatalf("expected 1 state, got %d", len(states))
}
st := states[0]
if st.Status != happydns.StatusWarn {
t.Errorf("status = %v, want Warn", st.Status)
}

View file

@ -86,9 +86,9 @@ func (p *whoisProvider) Collect(ctx context.Context, opts happydns.CheckerOption
}
// ExtractMetrics implements happydns.CheckerMetricsReporter.
func (p *whoisProvider) ExtractMetrics(raw json.RawMessage, collectedAt time.Time) ([]happydns.CheckMetric, error) {
func (p *whoisProvider) ExtractMetrics(ctx happydns.ReportContext, collectedAt time.Time) ([]happydns.CheckMetric, error) {
var data WHOISData
if err := json.Unmarshal(raw, &data); err != nil {
if err := json.Unmarshal(ctx.Data(), &data); err != nil {
return nil, err
}
@ -145,14 +145,14 @@ func (r *domainExpiryRule) ValidateOptions(opts happydns.CheckerOptions) error {
return nil
}
func (r *domainExpiryRule) Evaluate(ctx context.Context, obs happydns.ObservationGetter, opts happydns.CheckerOptions) happydns.CheckState {
func (r *domainExpiryRule) Evaluate(ctx context.Context, obs happydns.ObservationGetter, opts happydns.CheckerOptions) []happydns.CheckState {
var whois WHOISData
if err := obs.Get(ctx, ObservationKeyWhois, &whois); err != nil {
return happydns.CheckState{
return []happydns.CheckState{{
Status: happydns.StatusError,
Message: fmt.Sprintf("Failed to get WHOIS data: %v", err),
Code: "whois_error",
}
}}
}
// Read thresholds from options with defaults.
@ -163,29 +163,29 @@ func (r *domainExpiryRule) Evaluate(ctx context.Context, obs happydns.Observatio
meta := map[string]any{"days_remaining": daysRemaining}
if daysRemaining <= criticalDays {
return happydns.CheckState{
return []happydns.CheckState{{
Status: happydns.StatusCrit,
Message: fmt.Sprintf("Domain expires in %d days", daysRemaining),
Code: "expiry_critical",
Meta: meta,
}
}}
}
if daysRemaining <= warningDays {
return happydns.CheckState{
return []happydns.CheckState{{
Status: happydns.StatusWarn,
Message: fmt.Sprintf("Domain expires in %d days", daysRemaining),
Code: "expiry_warning",
Meta: meta,
}
}}
}
return happydns.CheckState{
return []happydns.CheckState{{
Status: happydns.StatusOK,
Message: fmt.Sprintf("Domain expires in %d days", daysRemaining),
Code: "expiry_ok",
Meta: meta,
}
}}
}
func init() {

View file

@ -27,6 +27,7 @@ import (
"testing"
"time"
sdk "git.happydns.org/checker-sdk-go/checker"
"git.happydns.org/happyDomain/model"
)
@ -57,7 +58,11 @@ func TestDomainExpiryRule_Evaluate(t *testing.T) {
ExpiryDate: now.Add(tc.expiresIn),
Registrar: "Test Registrar",
})
st := rule.Evaluate(context.Background(), obs, tc.opts)
states := rule.Evaluate(context.Background(), obs, tc.opts)
if len(states) != 1 {
t.Fatalf("expected 1 state, got %d", len(states))
}
st := states[0]
if st.Status != tc.want {
t.Errorf("status = %v, want %v (msg=%q)", st.Status, tc.want, st.Message)
}
@ -71,7 +76,11 @@ func TestDomainExpiryRule_Evaluate(t *testing.T) {
func TestDomainExpiryRule_EvaluateObservationError(t *testing.T) {
rule := &domainExpiryRule{}
obs := &stubObservationGetter{key: ObservationKeyWhois, err: errString("boom")}
st := rule.Evaluate(context.Background(), obs, nil)
states := rule.Evaluate(context.Background(), obs, nil)
if len(states) != 1 {
t.Fatalf("expected 1 state, got %d", len(states))
}
st := states[0]
if st.Status != happydns.StatusError {
t.Fatalf("expected StatusError, got %v", st.Status)
}
@ -116,7 +125,7 @@ func TestWhoisProvider_ExtractMetrics(t *testing.T) {
if err != nil {
t.Fatal(err)
}
metrics, err := p.ExtractMetrics(raw, collected)
metrics, err := p.ExtractMetrics(sdk.StaticReportContext(raw), collected)
if err != nil {
t.Fatal(err)
}

View file

@ -62,14 +62,14 @@ func (r *domainLockRule) ValidateOptions(opts happydns.CheckerOptions) error {
return fmt.Errorf("requiredStatuses must contain at least one EPP status code")
}
func (r *domainLockRule) Evaluate(ctx context.Context, obs happydns.ObservationGetter, opts happydns.CheckerOptions) happydns.CheckState {
func (r *domainLockRule) Evaluate(ctx context.Context, obs happydns.ObservationGetter, opts happydns.CheckerOptions) []happydns.CheckState {
var whois WHOISData
if err := obs.Get(ctx, ObservationKeyWhois, &whois); err != nil {
return happydns.CheckState{
return []happydns.CheckState{{
Status: happydns.StatusError,
Message: fmt.Sprintf("Failed to get WHOIS data: %v", err),
Code: "lock_error",
}
}}
}
requiredStr := defaultRequiredLockStatuses
@ -86,11 +86,11 @@ func (r *domainLockRule) Evaluate(ctx context.Context, obs happydns.ObservationG
}
if len(required) == 0 {
return happydns.CheckState{
return []happydns.CheckState{{
Status: happydns.StatusUnknown,
Message: "No required lock statuses configured",
Code: "lock_skipped",
}
}}
}
present := make(map[string]bool, len(whois.Status))
@ -106,7 +106,7 @@ func (r *domainLockRule) Evaluate(ctx context.Context, obs happydns.ObservationG
}
if len(missing) > 0 {
return happydns.CheckState{
return []happydns.CheckState{{
Status: happydns.StatusCrit,
Message: fmt.Sprintf("Missing lock status: %s", strings.Join(missing, ", ")),
Code: "lock_missing",
@ -114,17 +114,17 @@ func (r *domainLockRule) Evaluate(ctx context.Context, obs happydns.ObservationG
"missing": missing,
"present": whois.Status,
},
}
}}
}
return happydns.CheckState{
return []happydns.CheckState{{
Status: happydns.StatusOK,
Message: fmt.Sprintf("All required statuses present: %s", strings.Join(required, ", ")),
Code: "lock_ok",
Meta: map[string]any{
"required": required,
},
}
}}
}
func init() {

View file

@ -82,7 +82,11 @@ func TestDomainLockRule_Evaluate(t *testing.T) {
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
obs := newWhoisObs(&WHOISData{Status: tc.status})
st := rule.Evaluate(context.Background(), obs, tc.opts)
states := rule.Evaluate(context.Background(), obs, tc.opts)
if len(states) != 1 {
t.Fatalf("expected 1 state, got %d", len(states))
}
st := states[0]
if st.Status != tc.want {
t.Errorf("status = %v, want %v (msg=%q)", st.Status, tc.want, st.Message)
}
@ -107,7 +111,11 @@ func TestDomainLockRule_NilStatus(t *testing.T) {
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
obs := newWhoisObs(&WHOISData{Status: tc.status})
st := rule.Evaluate(context.Background(), obs, nil)
states := rule.Evaluate(context.Background(), obs, nil)
if len(states) != 1 {
t.Fatalf("expected 1 state, got %d", len(states))
}
st := states[0]
if st.Status != happydns.StatusCrit {
t.Errorf("status = %v, want Crit", st.Status)
}
@ -121,7 +129,11 @@ func TestDomainLockRule_NilStatus(t *testing.T) {
func TestDomainLockRule_EvaluateObservationError(t *testing.T) {
rule := &domainLockRule{}
obs := &stubObservationGetter{key: ObservationKeyWhois, err: errString("nope")}
st := rule.Evaluate(context.Background(), obs, nil)
states := rule.Evaluate(context.Background(), obs, nil)
if len(states) != 1 {
t.Fatalf("expected 1 state, got %d", len(states))
}
st := states[0]
if st.Status != happydns.StatusError || st.Code != "lock_error" {
t.Errorf("got %v / %q", st.Status, st.Code)
}

View file

@ -50,6 +50,10 @@ func (g *stubObservationGetter) Get(ctx context.Context, key happydns.Observatio
return json.Unmarshal(raw, dest)
}
func (g *stubObservationGetter) GetRelated(ctx context.Context, key happydns.ObservationKey) ([]happydns.RelatedObservation, error) {
return nil, nil
}
type errString string
func (e errString) Error() string { return string(e) }

32
checkers/tls.go Normal file
View file

@ -0,0 +1,32 @@
// This file is part of the happyDomain (R) project.
// Copyright (c) 2020-2026 happyDomain
// Authors: Pierre-Olivier Mercier, et al.
//
// This program is offered under a commercial and under the AGPL license.
// For commercial licensing, contact us at <contact@happydomain.org>.
//
// For AGPL licensing:
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
package checkers
import (
tls "git.happydns.org/checker-tls/checker"
"git.happydns.org/happyDomain/internal/checker"
)
func init() {
checker.RegisterObservationProvider(tls.Provider())
checker.RegisterExternalizableChecker(tls.Definition())
}

135
checks/deprecated_rr.go Normal file
View file

@ -0,0 +1,135 @@
// This file is part of the happyDomain (R) project.
// Copyright (c) 2020-2026 happyDomain
// Authors: Pierre-Olivier Mercier, et al.
//
// This program is offered under a commercial and under the AGPL license.
// For commercial licensing, contact us at <contact@happydomain.org>.
//
// For AGPL licensing:
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
package checks
import (
"context"
"encoding/json"
"fmt"
"strings"
"github.com/miekg/dns"
"git.happydns.org/happyDomain/model"
svcs "git.happydns.org/happyDomain/services"
)
func init() {
RegisterChecker("deprecated-records", &DeprecatedRecordCheck{})
}
// deprecatedTypes maps DNS record type numbers to a human-readable reason.
var deprecatedTypes = map[uint16]string{
dns.TypeSPF: "RFC 7208: use TXT instead",
38: "RFC 6563: use AAAA instead", // A6
dns.TypeNXT: "RFC 3755: use NSEC instead",
dns.TypeSIG: "RFC 3755: use RRSIG instead",
dns.TypeKEY: "RFC 3755: use DNSKEY instead",
11: "deprecated, not widely used", // WKS
}
// DeprecatedRecordFinding describes a single deprecated record type found.
type DeprecatedRecordFinding struct {
TypeName string `json:"type"`
Reason string `json:"reason"`
}
type DeprecatedRecordCheck struct{}
func (d *DeprecatedRecordCheck) ID() string {
return "deprecated-records"
}
func (d *DeprecatedRecordCheck) Name() string {
return "Deprecated DNS Record Types"
}
func (d *DeprecatedRecordCheck) Availability() happydns.CheckerAvailability {
return happydns.CheckerAvailability{
ApplyToService: true,
}
}
func (d *DeprecatedRecordCheck) Options() happydns.CheckerOptionsDocumentation {
return happydns.CheckerOptionsDocumentation{
RunOpts: []happydns.CheckerOptionDocumentation{
{
Id: "service",
Label: "Service",
AutoFill: happydns.AutoFillService,
},
},
}
}
func (d *DeprecatedRecordCheck) RunCheck(ctx context.Context, options happydns.CheckerOptions, meta map[string]string) (*happydns.CheckResult, error) {
service, ok := options["service"].(*happydns.ServiceMessage)
if !ok {
return nil, fmt.Errorf("service not defined")
}
svcBody, err := svcs.FindService(service.Type)
if err != nil {
return nil, fmt.Errorf("unknown service type %q: %w", service.Type, err)
}
if err := json.Unmarshal(service.Service, &svcBody); err != nil {
return nil, fmt.Errorf("failed to decode service: %w", err)
}
records, err := svcBody.GetRecords(service.Domain, service.Ttl, "")
if err != nil {
return nil, fmt.Errorf("failed to get records: %w", err)
}
// Collect unique deprecated types found.
seen := map[uint16]bool{}
var findings []DeprecatedRecordFinding
for _, rr := range records {
rrtype := rr.Header().Rrtype
if reason, deprecated := deprecatedTypes[rrtype]; deprecated && !seen[rrtype] {
seen[rrtype] = true
findings = append(findings, DeprecatedRecordFinding{
TypeName: dns.TypeToString[rrtype],
Reason: reason,
})
}
}
if len(findings) == 0 {
return &happydns.CheckResult{
Status: happydns.CheckResultStatusOK,
StatusLine: "No deprecated record types found",
Report: findings,
}, nil
}
typeNames := make([]string, len(findings))
for i, f := range findings {
typeNames[i] = f.TypeName
}
return &happydns.CheckResult{
Status: happydns.CheckResultStatusWarn,
StatusLine: "Deprecated record types found: " + strings.Join(typeNames, ", "),
Report: findings,
}, nil
}

414
checks/ns_restrictions.go Normal file
View file

@ -0,0 +1,414 @@
// This file is part of the happyDomain (R) project.
// Copyright (c) 2020-2026 happyDomain
// Authors: Pierre-Olivier Mercier, et al.
//
// This program is offered under a commercial and under the AGPL license.
// For commercial licensing, contact us at <contact@happydomain.org>.
//
// For AGPL licensing:
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
package checks
import (
"bytes"
"context"
"encoding/json"
"errors"
"fmt"
"html/template"
"net"
"strings"
"syscall"
"time"
"github.com/miekg/dns"
"git.happydns.org/happyDomain/model"
"git.happydns.org/happyDomain/services/abstract"
)
func init() {
RegisterChecker("ns_restrictions", &NSRestrictionsCheck{})
}
// NSRestrictionsReport contains the results of NS security restriction checks.
type NSRestrictionsReport struct {
Servers []NSServerResult `json:"servers"`
}
// NSServerResult holds the check results for a single nameserver IP.
type NSServerResult struct {
Name string `json:"name"`
Address string `json:"address"`
Checks []NSCheckItem `json:"checks"`
}
// NSCheckItem represents one security check for an NS server.
type NSCheckItem struct {
Name string `json:"name"`
OK bool `json:"ok"`
Detail string `json:"detail,omitempty"`
}
type NSRestrictionsCheck struct{}
func (c *NSRestrictionsCheck) ID() string {
return "ns_restrictions"
}
func (c *NSRestrictionsCheck) Name() string {
return "NS Security Restrictions"
}
func (c *NSRestrictionsCheck) Availability() happydns.CheckerAvailability {
return happydns.CheckerAvailability{
ApplyToService: true,
LimitToServices: []string{"abstract.Origin", "abstract.NSOnlyOrigin"},
}
}
func (c *NSRestrictionsCheck) Options() happydns.CheckerOptionsDocumentation {
return happydns.CheckerOptionsDocumentation{
RunOpts: []happydns.CheckerOptionDocumentation{
{
Id: "service",
Label: "Service",
AutoFill: happydns.AutoFillService,
},
{
Id: "domainName",
Label: "Domain name",
AutoFill: happydns.AutoFillDomainName,
},
},
}
}
// nsFromServiceOption extracts the list of NS records from an Origin or NSOnlyOrigin service.
func nsFromServiceOption(svc *happydns.ServiceMessage) []*dns.NS {
if svc.Type == "abstract.Origin" {
var origin abstract.Origin
if err := json.Unmarshal(svc.Service, &origin); err != nil {
return nil
}
return origin.NameServers
}
var origin abstract.NSOnlyOrigin
if err := json.Unmarshal(svc.Service, &origin); err != nil {
return nil
}
return origin.NameServers
}
// checkAXFR returns (ok bool, detail string).
// ok=false means the server accepted the zone transfer (CRITICAL).
func checkAXFR(ctx context.Context, domain, addr string) (bool, string) {
msg := new(dns.Msg)
msg.SetAxfr(dns.Fqdn(domain))
t := &dns.Transfer{}
t.DialTimeout = 5 * time.Second
t.ReadTimeout = 10 * time.Second
ch, err := t.In(msg, net.JoinHostPort(addr, "53"))
if err != nil {
// Connection refused or similar — transfer was refused, good.
return true, fmt.Sprintf("transfer refused: %s", err)
}
for env := range ch {
if env.Error != nil {
return true, fmt.Sprintf("transfer error: %s", env.Error)
}
for _, rr := range env.RR {
if rr.Header().Rrtype == dns.TypeSOA {
// Zone transfer succeeded — CRITICAL.
return false, "AXFR zone transfer accepted"
}
}
}
return true, "AXFR refused"
}
// checkIXFR returns (ok bool, detail string).
// ok=false means the server answered with records (WARN).
func checkIXFR(ctx context.Context, domain, addr string) (bool, string) {
msg := new(dns.Msg)
msg.SetIxfr(dns.Fqdn(domain), 0, "", "")
cl := &dns.Client{Net: "udp", Timeout: 5 * time.Second}
resp, _, err := cl.ExchangeContext(ctx, msg, net.JoinHostPort(addr, "53"))
if err != nil {
return true, fmt.Sprintf("query failed: %s", err)
}
if resp.Rcode != dns.RcodeSuccess {
return true, fmt.Sprintf("IXFR refused (rcode=%s)", dns.RcodeToString[resp.Rcode])
}
if len(resp.Answer) > 0 {
return false, fmt.Sprintf("IXFR accepted with %d answer(s)", len(resp.Answer))
}
return true, "IXFR refused or empty"
}
// checkNoRecursion returns (ok bool, detail string).
// ok=false means the server offers recursion (WARN).
func checkNoRecursion(ctx context.Context, domain, addr string) (bool, string) {
msg := new(dns.Msg)
msg.SetQuestion(dns.Fqdn(domain), dns.TypeSOA)
msg.RecursionDesired = true
cl := &dns.Client{Net: "udp", Timeout: 5 * time.Second}
resp, _, err := cl.ExchangeContext(ctx, msg, net.JoinHostPort(addr, "53"))
if err != nil {
return true, fmt.Sprintf("query failed: %s", err)
}
if resp.RecursionAvailable {
return false, "recursion available (RA bit set)"
}
return true, "recursion not available"
}
// checkANYHandled returns (ok bool, detail string).
// ok=false means the server returned a full record set for ANY (WARN).
// Per RFC 8482, servers should return HINFO or minimal response.
func checkANYHandled(ctx context.Context, domain, addr string) (bool, string) {
msg := new(dns.Msg)
msg.SetQuestion(dns.Fqdn(domain), dns.TypeANY)
cl := &dns.Client{Net: "udp", Timeout: 5 * time.Second}
resp, _, err := cl.ExchangeContext(ctx, msg, net.JoinHostPort(addr, "53"))
if err != nil {
return true, fmt.Sprintf("query failed: %s", err)
}
if resp.Rcode != dns.RcodeSuccess {
return true, fmt.Sprintf("ANY refused (rcode=%s)", dns.RcodeToString[resp.Rcode])
}
// If there's only a HINFO record, it's RFC 8482 compliant.
if len(resp.Answer) == 1 {
if _, ok := resp.Answer[0].(*dns.HINFO); ok {
return true, "RFC 8482 compliant HINFO response"
}
}
// Empty answer or TC (truncated) with no answers — also acceptable.
if len(resp.Answer) == 0 {
return true, "ANY returned empty answer"
}
return false, fmt.Sprintf("ANY returned %d records (not RFC 8482 compliant)", len(resp.Answer))
}
// checkIsAuthoritative returns (ok bool, detail string).
// ok=false means the server is not authoritative for the zone (INFO).
func checkIsAuthoritative(ctx context.Context, domain, addr string) (bool, string) {
msg := new(dns.Msg)
msg.SetQuestion(dns.Fqdn(domain), dns.TypeSOA)
cl := &dns.Client{Net: "udp", Timeout: 5 * time.Second}
resp, _, err := cl.ExchangeContext(ctx, msg, net.JoinHostPort(addr, "53"))
if err != nil {
return false, fmt.Sprintf("query failed: %s", err)
}
if resp.Authoritative {
return true, "server is authoritative (AA bit set)"
}
return false, "server is not authoritative (AA bit not set)"
}
// checkServerAddr runs all NS security checks against a single IP address.
// Returns the result and the worst status encountered.
func checkServerAddr(ctx context.Context, domain, nsHost, addr string) (NSServerResult, happydns.CheckResultStatus) {
result := NSServerResult{Name: nsHost, Address: addr}
status := happydns.CheckResultStatusOK
type checkDef struct {
name string
fn func(context.Context, string, string) (bool, string)
failLevel happydns.CheckResultStatus
}
checks := []checkDef{
{"AXFR refused", checkAXFR, happydns.CheckResultStatusCritical},
{"IXFR refused", checkIXFR, happydns.CheckResultStatusWarn},
{"No recursion", checkNoRecursion, happydns.CheckResultStatusWarn},
{"ANY handled (RFC 8482)", checkANYHandled, happydns.CheckResultStatusWarn},
{"Is authoritative", checkIsAuthoritative, happydns.CheckResultStatusInfo},
}
for _, ch := range checks {
ok, detail := ch.fn(ctx, domain, addr)
result.Checks = append(result.Checks, NSCheckItem{Name: ch.name, OK: ok, Detail: detail})
if !ok && status > ch.failLevel {
status = ch.failLevel
}
}
return result, status
}
// checkNameServer resolves nsHost and runs checks on each address.
// Returns results and summary parts for each address.
func checkNameServer(ctx context.Context, domain, nsHost string) ([]NSServerResult, []string, happydns.CheckResultStatus) {
worstStatus := happydns.CheckResultStatusOK
addrs, err := net.LookupHost(nsHost)
if err != nil {
return []NSServerResult{{
Name: nsHost,
Address: "",
Checks: []NSCheckItem{{Name: "DNS resolution", OK: false, Detail: fmt.Sprintf("lookup failed: %s", err)}},
}}, []string{fmt.Sprintf("%s: resolution failed", nsHost)}, happydns.CheckResultStatusWarn
}
var results []NSServerResult
var summaryParts []string
for _, addr := range addrs {
// Skip IPv6 addresses when there is no IPv6 connectivity.
if ip := net.ParseIP(addr); ip != nil && ip.To4() == nil {
conn, err := net.DialTimeout("udp", net.JoinHostPort(addr, "53"), 3*time.Second)
if errors.Is(err, syscall.ENETUNREACH) {
results = append(results, NSServerResult{
Name: nsHost,
Address: addr,
Checks: []NSCheckItem{{Name: "IPv6 connectivity", Detail: "unable to test due to the lack of IPv6 connectivity"}},
})
summaryParts = append(summaryParts, fmt.Sprintf("%s (%s): skipped (no IPv6)", nsHost, addr))
continue
}
if conn != nil {
conn.Close()
}
}
serverResult, serverStatus := checkServerAddr(ctx, domain, nsHost, addr)
results = append(results, serverResult)
if serverStatus < worstStatus {
worstStatus = serverStatus
}
switch serverStatus {
case happydns.CheckResultStatusCritical:
summaryParts = append(summaryParts, fmt.Sprintf("%s (%s): CRITICAL", nsHost, addr))
case happydns.CheckResultStatusWarn:
summaryParts = append(summaryParts, fmt.Sprintf("%s (%s): WARN", nsHost, addr))
default:
summaryParts = append(summaryParts, fmt.Sprintf("%s (%s): OK", nsHost, addr))
}
}
return results, summaryParts, worstStatus
}
func (c *NSRestrictionsCheck) RunCheck(ctx context.Context, options happydns.CheckerOptions, meta map[string]string) (*happydns.CheckResult, error) {
service, ok := options["service"].(*happydns.ServiceMessage)
if !ok {
return nil, fmt.Errorf("service not defined")
}
if service.Type != "abstract.Origin" && service.Type != "abstract.NSOnlyOrigin" {
return nil, fmt.Errorf("service is %s, expected abstract.Origin or abstract.NSOnlyOrigin", service.Type)
}
domainName := ""
if dn, ok := options["domainName"].(string); ok {
domainName = dn
}
if domainName == "" {
domainName = service.Domain
}
nameServers := nsFromServiceOption(service)
if len(nameServers) == 0 {
return nil, fmt.Errorf("no nameservers found in service")
}
report := NSRestrictionsReport{}
overallStatus := happydns.CheckResultStatusOK
var summaryParts []string
for _, ns := range nameServers {
nsHost := strings.TrimSuffix(ns.Ns, ".")
results, parts, status := checkNameServer(ctx, domainName, nsHost)
report.Servers = append(report.Servers, results...)
summaryParts = append(summaryParts, parts...)
if status < overallStatus {
overallStatus = status
}
}
return &happydns.CheckResult{
Status: overallStatus,
StatusLine: strings.Join(summaryParts, " | "),
Report: report,
}, nil
}
var nsRestrictionsHTMLTemplate = template.Must(template.New("ns_restrictions").Parse(`<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title>NS Security Restrictions</title>
<style>
body { font-family: sans-serif; margin: 1em; }
table { border-collapse: collapse; width: 100%; margin-bottom: 1.5em; }
th, td { border: 1px solid #ccc; padding: 0.4em 0.8em; text-align: left; }
th { background: #f0f0f0; }
.ok { color: #2a7a2a; font-weight: bold; }
.fail { color: #c0392b; font-weight: bold; }
h2 { margin-top: 1.5em; }
</style>
</head>
<body>
<h1>NS Security Restrictions Report</h1>
{{range .Servers}}
<h2>{{.Name}} ({{.Address}})</h2>
<table>
<thead><tr><th>Check</th><th>Result</th><th>Detail</th></tr></thead>
<tbody>
{{range .Checks}}
<tr>
<td>{{.Name}}</td>
<td>{{if .OK}}<span class="ok">&#10003; OK</span>{{else}}<span class="fail">&#10007; FAIL</span>{{end}}</td>
<td>{{.Detail}}</td>
</tr>
{{end}}
</tbody>
</table>
{{end}}
</body>
</html>
`))
// GetHTMLReport implements happydns.CheckerHTMLReporter.
func (c *NSRestrictionsCheck) GetHTMLReport(raw json.RawMessage) (string, error) {
var report NSRestrictionsReport
if err := json.Unmarshal(raw, &report); err != nil {
return "", fmt.Errorf("failed to parse report: %w", err)
}
var buf bytes.Buffer
if err := nsRestrictionsHTMLTemplate.Execute(&buf, report); err != nil {
return "", fmt.Errorf("failed to render template: %w", err)
}
return buf.String(), nil
}

67
docs/checker-discovery.md Normal file
View file

@ -0,0 +1,67 @@
# Cross-checker discovery
This document describes the contract between the SDK types (`DiscoveryEntry`,
`DiscoveryPublisher`, `RelatedObservation`, `ObservationGetter.GetRelated`,
`ReportContext`) and the happyDomain host that implements the scheduler and
storage behind them.
It exists because the SDK alone does not answer the interesting questions:
*who* stores entries, *when* are related observations resolved, what happens
when a consumer is missing, how stale data is pruned, and so on. Checker
authors need to know this to write code that behaves correctly at the edges.
## Model
```
┌──────────────┐ Collect ┌─────────────┐
│ producer A │ ───────────────▶ │ host │
│ │ ◀─── entries ─── │ (scheduler │
└──────────────┘ │ + store) │
│ │
┌──────────────┐ Collect (fed │ │
│ consumer B │ entries via │ │
│ │ AutoFill…) ◀│ │
│ │ ─── observation─▶│ │
└──────────────┘ └─────────────┘
GetRelated / ReportContext.Related
on producer A's next evaluate / report
```
Two independent flows composed by the host:
1. **Publication.** `DiscoveryPublisher.DiscoverEntries` returns a set of
`DiscoveryEntry` at the end of each `Collect`. The host replaces the
previous set for `(producer, target)` atomically. Entries are opaque
to the host beyond `(Type, Ref)`.
2. **Observation lineage.** When a consumer checker runs on the same target
and its option `AutoFillDiscoveryEntries` is populated, the host passes
it the entries it knows about. The consumer filters by `Type`, reads
`Payload` under the corresponding contract, produces its observation,
and includes per-entry references (matching `DiscoveryEntry.Ref`) in its
output. The host indexes those references so that a subsequent
`GetRelated` / `ReportContext.Related` call from the original producer
can return them.
## Host responsibilities
- **Entry index.** Store entries keyed by `(producer checker id, target,
entry type, ref)`. On each successful collection, compute the diff vs.
the previous set and apply it atomically. The visible state must never be
a mix of old and new entries.
- **Observation → entry linkage.** When a consumer stores an observation
on behalf of a `Ref`, record that linkage. The producer's next
`GetRelated(key)` query reads this index.
- **Filtering at `AutoFillDiscoveryEntries` fill time.** Give the consumer
all entries known for the target, not a pre-filtered subset. The SDK does
not know which types the consumer understands; the consumer is the only
place that knows its own contract.
- **Garbage collection.** When an entry disappears from the producer's
latest set, the observations that covered it become stale. Drop them at
the next consumer cycle or keep them with a TTL — either is acceptable,
but `GetRelated` must not return observations whose `Ref` no longer
exists.
- **`CollectedAt` fidelity.** `RelatedObservation.CollectedAt` must be
populated by the host so reporters can decide whether to trust stale
data.

3
go.mod
View file

@ -8,7 +8,8 @@ require (
git.happydns.org/checker-matrix v0.0.0-20260407211824-2bb91d33d489
git.happydns.org/checker-ns-restrictions v0.0.0-20260415205411-f1e3096f606d
git.happydns.org/checker-ping v0.0.0-20260407194626-a2ebf17774fc
git.happydns.org/checker-sdk-go v0.5.0
git.happydns.org/checker-sdk-go v1.2.0
git.happydns.org/checker-tls v0.2.0
git.happydns.org/checker-zonemaster v0.0.0-20260407202727-979757b5a8fc
github.com/JGLTechnologies/gin-rate-limit v1.5.8
github.com/StackExchange/dnscontrol/v4 v4.34.0

13
go.sum
View file

@ -14,8 +14,10 @@ git.happydns.org/checker-ns-restrictions v0.0.0-20260415205411-f1e3096f606d h1:W
git.happydns.org/checker-ns-restrictions v0.0.0-20260415205411-f1e3096f606d/go.mod h1:Sw8SqlrTAi3ZSVQQ+5kKE8reekxvLRm7oYlUYfJD+Z4=
git.happydns.org/checker-ping v0.0.0-20260407194626-a2ebf17774fc h1:jKEOx2NDbHHxjCy1fUkcn1RgpzOKbE+bGRsF+ITNigI=
git.happydns.org/checker-ping v0.0.0-20260407194626-a2ebf17774fc/go.mod h1:wphWmslFhKcpWfJTrHdChv8DkhUP9jwis7V2jy7vOX0=
git.happydns.org/checker-sdk-go v0.5.0 h1:wpFIK/vxanrAYf1OlewSnSCYc7KOJKdu88uUWB7HIQI=
git.happydns.org/checker-sdk-go v0.5.0/go.mod h1:aNAcfYFfbhvH9kJhE0Njp5GX0dQbxdRB0rJ0KvSC5nI=
git.happydns.org/checker-sdk-go v1.2.0 h1:v4MpKAz0W3PwP+bxx3pya8w893sVH5xTD1of1cc0TV8=
git.happydns.org/checker-sdk-go v1.2.0/go.mod h1:aNAcfYFfbhvH9kJhE0Njp5GX0dQbxdRB0rJ0KvSC5nI=
git.happydns.org/checker-tls v0.2.0 h1:2dYpcePBylUc3le76fFlLbxraiLpGESmOhx4NfD7REM=
git.happydns.org/checker-tls v0.2.0/go.mod h1:0ZSG0CTP007SHBPE7qInESVIOcW+xgucHUhHgj6MeZ8=
git.happydns.org/checker-zonemaster v0.0.0-20260407202727-979757b5a8fc h1:y5xjoqLA/WztFWhEUifOwnJ6POjl+Udw6bWjzQ2afOw=
git.happydns.org/checker-zonemaster v0.0.0-20260407202727-979757b5a8fc/go.mod h1:B1P23OMm82GfAtYw8vCbspc7qULsFA0u/tqR+SGAaNw=
github.com/Azure/azure-sdk-for-go/sdk/azcore v1.21.0 h1:fou+2+WFTib47nS+nz/ozhEBnvU96bKHy6LjRsY4E28=
@ -135,6 +137,10 @@ github.com/bmatcuk/doublestar v1.1.1/go.mod h1:UD6OnuiIn0yFxxA2le/rnRU1G4RaI4UvF
github.com/boombuler/barcode v1.0.1-0.20190219062509-6c824513bacc/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8=
github.com/boombuler/barcode v1.1.0 h1:ChaYjBR63fr4LFyGn8E8nt7dBSt3MiU3zMOZqFvVkHo=
github.com/boombuler/barcode v1.1.0/go.mod h1:paBWMcWSl3LHKBqUq+rly7CNSldXjb2rDl3JlRe0mD8=
github.com/bsm/ginkgo/v2 v2.12.0 h1:Ny8MWAHyOepLGlLKYmXG4IEkioBysk6GpaRTLC8zwWs=
github.com/bsm/ginkgo/v2 v2.12.0/go.mod h1:SwYbGRRDovPVboqFv0tPTcG1sN61LM1Z4ARdbAV9g4c=
github.com/bsm/gomega v1.27.10 h1:yeMWxP2pV2fG3FgAODIY8EiRE3dy0aeFYt4l7wh6yKA=
github.com/bsm/gomega v1.27.10/go.mod h1:JyEr/xRbxbtgWNi8tIEVPUYZ5Dzef52k01W3YH0H+O0=
github.com/bytedance/gopkg v0.1.4 h1:oZnQwnX82KAIWb7033bEwtxvTqXcYMxDBaQxo5JJHWM=
github.com/bytedance/gopkg v0.1.4/go.mod h1:v1zWfPm21Fb+OsyXN2VAHdL6TBb2L88anLQgdyje6R4=
github.com/bytedance/sonic v1.15.0 h1:/PXeWFaR5ElNcVE84U0dOHjiMHQOwNIx3K4ymzh/uSE=
@ -629,6 +635,8 @@ github.com/yuin/goldmark v1.4.1/go.mod h1:mwnBkeHKe2W/ZEtQ+71ViKU8L12m81fl3OWwC1
github.com/yuin/goldmark v1.4.13/go.mod h1:6yULJ656Px+3vBD8DxQVa3kxgyrAnzto9xy5taEt/CY=
github.com/yuin/goldmark v1.8.2 h1:kEGpgqJXdgbkhcOgBxkC0X0PmoPG1ZyoZ117rDVp4zE=
github.com/yuin/goldmark v1.8.2/go.mod h1:ip/1k0VRfGynBgxOz0yCqHrbZXhcjxyuS66Brc7iBKg=
github.com/zeebo/xxh3 v1.0.2 h1:xZmwmqxHZA8AI603jOQ0tMqmBr9lPeFwGg6d+xy9DC0=
github.com/zeebo/xxh3 v1.0.2/go.mod h1:5NWz9Sef7zIDm2JHfFlcQvNekmcEl9ekUZQQKCYaDcA=
go.mongodb.org/mongo-driver v1.13.1/go.mod h1:wcDf1JBCXy2mOW0bWHwO/IOYqdca1MPCwDtFu/Z9+eo=
go.mongodb.org/mongo-driver v1.17.9 h1:IexDdCuuNJ3BHrELgBlyaH9p60JXAvdzWR128q+U5tU=
go.mongodb.org/mongo-driver v1.17.9/go.mod h1:LlOhpH5NUEfhxcAwG0UEkMqwYcc4JU18gtCdGudk/tQ=
@ -648,7 +656,6 @@ go.opentelemetry.io/otel/sdk/metric v1.43.0 h1:S88dyqXjJkuBNLeMcVPRFXpRw2fuwdvfC
go.opentelemetry.io/otel/sdk/metric v1.43.0/go.mod h1:C/RJtwSEJ5hzTiUz5pXF1kILHStzb9zFlIEe85bhj6A=
go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09nk+3A=
go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0=
go.uber.org/atomic v1.9.0 h1:ECmE8Bn/WFTYwEW/bpKD3M8VtR/zQVbavAoalC1PYyE=
go.uber.org/atomic v1.9.0/go.mod h1:fEN4uk6kAWBTFdckzkM89CLk9XfWZrxpCo0nPH17wJc=
go.uber.org/atomic v1.11.0 h1:ZvwS0R+56ePWxUNi+Atn9dWONBPp/AUETXlHW0DxSjE=
go.uber.org/atomic v1.11.0/go.mod h1:LUxbIzbOniOlMKjJjyPfpl4v+PKK2cNJn91OQbhoJI0=

View file

@ -160,6 +160,26 @@ func (bc *BackupController) DoBackup() (ret happydns.Backup) {
}
}
// Discovery entries.
if entryIter, err := bc.store.ListAllDiscoveryEntries(); err != nil {
ret.Errors = append(ret.Errors, fmt.Sprintf("unable to retrieve DiscoveryEntries: %s", err.Error()))
} else {
defer entryIter.Close()
for entryIter.Next() {
ret.DiscoveryEntries = append(ret.DiscoveryEntries, entryIter.Item())
}
}
// Discovery observation refs.
if refIter, err := bc.store.ListAllDiscoveryObservationRefs(); err != nil {
ret.Errors = append(ret.Errors, fmt.Sprintf("unable to retrieve DiscoveryObservationRefs: %s", err.Error()))
} else {
defer refIter.Close()
for refIter.Next() {
ret.DiscoveryObservationRefs = append(ret.DiscoveryObservationRefs, refIter.Item())
}
}
return
}
@ -238,6 +258,18 @@ func (bc *BackupController) DoRestore(backup *happydns.Backup) (errs error) {
errs = errors.Join(errs, bc.store.RestoreExecution(exec))
}
// Discovery entries. Restored after snapshots (referenced indirectly via
// target + producer, no FK), before observation refs which carry snapshot
// pointers that must resolve at lookup time.
for _, entry := range backup.DiscoveryEntries {
errs = errors.Join(errs, bc.store.RestoreDiscoveryEntry(entry))
}
// Discovery observation refs.
for _, ref := range backup.DiscoveryObservationRefs {
errs = errors.Join(errs, bc.store.RestoreDiscoveryObservationRef(ref))
}
return
}

View file

@ -22,6 +22,7 @@
package controller
import (
"encoding/json"
"fmt"
"net/http"
@ -33,6 +34,20 @@ import (
"git.happydns.org/happyDomain/model"
)
// buildReportContext returns a ReportContext for the given primary payload.
// When the engine exposes a related-observation lookup, the context resolves
// Related(key) against discovery storage; otherwise a static context is
// returned.
func (cc *CheckerController) buildReportContext(c *gin.Context, checkerID string, target happydns.CheckTarget, raw json.RawMessage) happydns.ReportContext {
var lookup checkerPkg.RelatedObservationLookup
if r, ok := cc.engine.(interface {
RelatedLookup() checkerPkg.RelatedObservationLookup
}); ok {
lookup = r.RelatedLookup()
}
return checkerPkg.BuildReportContext(c.Request.Context(), checkerID, target, raw, lookup)
}
// ExecutionHandler is a middleware that validates the executionId path parameter,
// checks target scope, and sets "execution" in context.
func (cc *CheckerController) ExecutionHandler(c *gin.Context) {
@ -254,7 +269,7 @@ func (cc *CheckerController) GetExecutionResult(c *gin.Context) {
ruleName := c.Param("ruleName")
for _, state := range eval.States {
if state.Code == ruleName {
if state.RuleName == ruleName {
c.JSON(http.StatusOK, state)
return
}
@ -291,7 +306,8 @@ func (cc *CheckerController) GetExecutionHTMLReport(c *gin.Context) {
return
}
htmlContent, supported, err := checkerPkg.GetHTMLReport(obsKey, val)
rc := cc.buildReportContext(c, exec.CheckerID, targetFromContext(c), val)
htmlContent, supported, err := checkerPkg.GetHTMLReport(obsKey, rc)
if err != nil {
middleware.ErrorResponse(c, http.StatusInternalServerError, err)
return

View file

@ -99,7 +99,7 @@ func (p *testHTMLObservationProvider) Key() happydns.ObservationKey { return "te
func (p *testHTMLObservationProvider) Collect(ctx context.Context, opts happydns.CheckerOptions) (any, error) {
return map[string]any{"html": true}, nil
}
func (p *testHTMLObservationProvider) GetHTMLReport(raw json.RawMessage) (string, error) {
func (p *testHTMLObservationProvider) GetHTMLReport(ctx happydns.ReportContext) (string, error) {
return "<html><body>test report</body></html>", nil
}
@ -111,8 +111,8 @@ type testCheckRule struct {
func (r *testCheckRule) Name() string { return r.name }
func (r *testCheckRule) Description() string { return "test rule: " + r.name }
func (r *testCheckRule) Evaluate(ctx context.Context, obs happydns.ObservationGetter, opts happydns.CheckerOptions) happydns.CheckState {
return happydns.CheckState{Status: r.status, Code: r.name}
func (r *testCheckRule) Evaluate(ctx context.Context, obs happydns.ObservationGetter, opts happydns.CheckerOptions) []happydns.CheckState {
return []happydns.CheckState{{Status: r.status}}
}
// registerTestChecker registers a checker for controller tests and returns its ID.

View file

@ -277,12 +277,15 @@ func (app *App) initUsecases() {
app.usecases.checkerOptionsUC = checkerUC.NewCheckerOptionsUsecase(app.store, app.store)
app.usecases.checkerPlanUC = checkerUC.NewCheckPlanUsecase(app.store)
app.usecases.checkerStatusUC = checkerUC.NewCheckStatusUsecase(app.store, app.store, app.store, app.store)
app.usecases.checkerOptionsUC.WithDiscoveryEntryStore(app.store)
app.usecases.checkerEngine = checkerUC.NewCheckerEngine(
app.usecases.checkerOptionsUC,
app.store,
app.store,
app.store,
app.store,
app.store,
app.store,
)
// Build the user-level gate so paused or long-inactive users do not
// get checked. The same user resolver is reused by the janitor for

View file

@ -60,6 +60,16 @@ func (s *instrumentedStorage) ClearCheckerConfigurations() (err error) {
return s.inner.ClearCheckerConfigurations()
}
func (s *instrumentedStorage) ClearDiscoveryEntries() (err error) {
defer observe("delete", "discovery_entry")(&err)
return s.inner.ClearDiscoveryEntries()
}
func (s *instrumentedStorage) ClearDiscoveryObservationRefs() (err error) {
defer observe("delete", "discovery_observation")(&err)
return s.inner.ClearDiscoveryObservationRefs()
}
func (s *instrumentedStorage) ClearDomains() (err error) {
defer observe("delete", "domain")(&err)
return s.inner.ClearDomains()
@ -187,6 +197,16 @@ func (s *instrumentedStorage) DeleteCheckerConfiguration(checkerName string, use
return s.inner.DeleteCheckerConfiguration(checkerName, userId, domainId, serviceId)
}
func (s *instrumentedStorage) DeleteDiscoveryEntriesByProducer(producerID string, target happydns.CheckTarget) (err error) {
defer observe("delete", "discovery_entry")(&err)
return s.inner.DeleteDiscoveryEntriesByProducer(producerID, target)
}
func (s *instrumentedStorage) DeleteDiscoveryObservationRefsForSnapshot(snapshotID happydns.Identifier) (err error) {
defer observe("delete", "discovery_observation")(&err)
return s.inner.DeleteDiscoveryObservationRefsForSnapshot(snapshotID)
}
func (s *instrumentedStorage) DeleteDomain(domainid happydns.Identifier) (err error) {
defer observe("delete", "domain")(&err)
return s.inner.DeleteDomain(domainid)
@ -362,6 +382,16 @@ func (s *instrumentedStorage) ListAllCheckerConfigurations() (ret happydns.Itera
return s.inner.ListAllCheckerConfigurations()
}
func (s *instrumentedStorage) ListAllDiscoveryEntries() (ret happydns.Iterator[happydns.StoredDiscoveryEntry], err error) {
defer observe("list", "discovery_entry")(&err)
return s.inner.ListAllDiscoveryEntries()
}
func (s *instrumentedStorage) ListAllDiscoveryObservationRefs() (ret happydns.Iterator[happydns.DiscoveryObservationRef], err error) {
defer observe("list", "discovery_observation")(&err)
return s.inner.ListAllDiscoveryObservationRefs()
}
func (s *instrumentedStorage) ListAllDomainLogs() (ret happydns.Iterator[happydns.DomainLogWithDomainId], err error) {
defer observe("list", "domain_log")(&err)
return s.inner.ListAllDomainLogs()
@ -432,6 +462,21 @@ func (s *instrumentedStorage) ListCheckerConfiguration(checkerName string) (ret
return s.inner.ListCheckerConfiguration(checkerName)
}
func (s *instrumentedStorage) ListDiscoveryEntriesByProducer(producerID string, target happydns.CheckTarget) (ret []*happydns.StoredDiscoveryEntry, err error) {
defer observe("list", "discovery_entry")(&err)
return s.inner.ListDiscoveryEntriesByProducer(producerID, target)
}
func (s *instrumentedStorage) ListDiscoveryEntriesByTarget(target happydns.CheckTarget) (ret []*happydns.StoredDiscoveryEntry, err error) {
defer observe("list", "discovery_entry")(&err)
return s.inner.ListDiscoveryEntriesByTarget(target)
}
func (s *instrumentedStorage) ListDiscoveryObservationRefs(producerID string, target happydns.CheckTarget, ref string) (ret []*happydns.DiscoveryObservationRef, err error) {
defer observe("list", "discovery_observation")(&err)
return s.inner.ListDiscoveryObservationRefs(producerID, target, ref)
}
func (s *instrumentedStorage) ListDomainLogs(domain *happydns.Domain) (ret []*happydns.DomainLog, err error) {
defer observe("list", "domain_log")(&err)
return s.inner.ListDomainLogs(domain)
@ -489,11 +534,31 @@ func (s *instrumentedStorage) PutCachedObservation(target happydns.CheckTarget,
return s.inner.PutCachedObservation(target, key, entry)
}
func (s *instrumentedStorage) PutDiscoveryObservationRef(ref *happydns.DiscoveryObservationRef) (err error) {
defer observe("put", "discovery_observation")(&err)
return s.inner.PutDiscoveryObservationRef(ref)
}
func (s *instrumentedStorage) ReplaceDiscoveryEntries(producerID string, target happydns.CheckTarget, entries []happydns.DiscoveryEntry) (err error) {
defer observe("update", "discovery_entry")(&err)
return s.inner.ReplaceDiscoveryEntries(producerID, target, entries)
}
func (s *instrumentedStorage) RestoreCheckPlan(plan *happydns.CheckPlan) (err error) {
defer observe("restore", "check_plan")(&err)
return s.inner.RestoreCheckPlan(plan)
}
func (s *instrumentedStorage) RestoreDiscoveryEntry(entry *happydns.StoredDiscoveryEntry) (err error) {
defer observe("restore", "discovery_entry")(&err)
return s.inner.RestoreDiscoveryEntry(entry)
}
func (s *instrumentedStorage) RestoreDiscoveryObservationRef(ref *happydns.DiscoveryObservationRef) (err error) {
defer observe("restore", "discovery_observation")(&err)
return s.inner.RestoreDiscoveryObservationRef(ref)
}
func (s *instrumentedStorage) RestoreEvaluation(eval *happydns.CheckEvaluation) (err error) {
defer observe("restore", "check_evaluation")(&err)
return s.inner.RestoreEvaluation(eval)
@ -504,7 +569,6 @@ func (s *instrumentedStorage) RestoreExecution(exec *happydns.Execution) (err er
return s.inner.RestoreExecution(exec)
}
func (s *instrumentedStorage) SchemaVersion() int { return s.inner.SchemaVersion() }
func (s *instrumentedStorage) SetLastSchedulerRun(t time.Time) (err error) {

View file

@ -44,6 +44,7 @@ import (
"encoding/json"
"errors"
"fmt"
"log"
"sync"
"time"
@ -55,6 +56,11 @@ import (
// Returns the raw data and collection time, or an error if not cached.
type ObservationCacheLookup func(target happydns.CheckTarget, key happydns.ObservationKey) (json.RawMessage, time.Time, error)
// RelatedObservationLookup resolves observations produced by other checkers
// on DiscoveryEntry records this checker has published for the given target.
// It returns the empty slice (not an error) when there is nothing to relate.
type RelatedObservationLookup func(ctx context.Context, producerCheckerID string, target happydns.CheckTarget, key happydns.ObservationKey) ([]happydns.RelatedObservation, error)
// ObservationContext provides lazy-loading, cached, thread-safe access to observation data.
// Collected data is serialized to json.RawMessage immediately after collection.
//
@ -65,15 +71,18 @@ type ObservationCacheLookup func(target happydns.CheckTarget, key happydns.Obser
// installs an inflight channel, runs the collection, then closes the
// channel; the others wait on it and read the cached result afterwards.
type ObservationContext struct {
target happydns.CheckTarget
opts happydns.CheckerOptions
cache map[happydns.ObservationKey]json.RawMessage
errors map[happydns.ObservationKey]error
inflight map[happydns.ObservationKey]chan struct{}
mu sync.Mutex
cacheLookup ObservationCacheLookup // nil = no DB cache
freshness time.Duration // 0 = always collect
providerOverride map[happydns.ObservationKey]happydns.ObservationProvider
target happydns.CheckTarget
opts happydns.CheckerOptions
cache map[happydns.ObservationKey]json.RawMessage
errors map[happydns.ObservationKey]error
inflight map[happydns.ObservationKey]chan struct{}
entries map[happydns.ObservationKey][]happydns.DiscoveryEntry
mu sync.Mutex
cacheLookup ObservationCacheLookup // nil = no DB cache
freshness time.Duration // 0 = always collect
providerOverride map[happydns.ObservationKey]happydns.ObservationProvider
producerCheckerID string // filled by engine; empty = GetRelated always empty
relatedLookup RelatedObservationLookup // nil = GetRelated always empty
}
// NewObservationContext creates a new ObservationContext for the given target and options.
@ -86,11 +95,27 @@ func NewObservationContext(target happydns.CheckTarget, opts happydns.CheckerOpt
cache: make(map[happydns.ObservationKey]json.RawMessage),
errors: make(map[happydns.ObservationKey]error),
inflight: make(map[happydns.ObservationKey]chan struct{}),
entries: make(map[happydns.ObservationKey][]happydns.DiscoveryEntry),
cacheLookup: cacheLookup,
freshness: freshness,
}
}
// Entries returns the DiscoveryEntry records published by each observation
// provider during this run, aggregated by observation key. The engine
// collects these after rule evaluation to persist them via the discovery
// store.
func (oc *ObservationContext) Entries() map[happydns.ObservationKey][]happydns.DiscoveryEntry {
oc.mu.Lock()
defer oc.mu.Unlock()
out := make(map[happydns.ObservationKey][]happydns.DiscoveryEntry, len(oc.entries))
for k, v := range oc.entries {
out[k] = append([]happydns.DiscoveryEntry(nil), v...)
}
return out
}
// SetProviderOverride registers a per-context provider that takes precedence
// over the global registry for the given observation key. This is used to
// substitute local providers with HTTP-backed ones when an endpoint is configured.
@ -198,6 +223,16 @@ func (oc *ObservationContext) collect(ctx context.Context, key happydns.Observat
return nil, err
}
if pub, ok := provider.(sdk.DiscoveryPublisher); ok {
if entries, err := pub.DiscoverEntries(val); err != nil {
log.Printf("observation %q: DiscoverEntries failed: %v", key, err)
} else if len(entries) > 0 {
oc.mu.Lock()
oc.entries[key] = entries
oc.mu.Unlock()
}
}
raw, err := json.Marshal(val)
if err != nil {
return nil, fmt.Errorf("observation %q: marshal failed: %w", key, err)
@ -205,6 +240,32 @@ func (oc *ObservationContext) collect(ctx context.Context, key happydns.Observat
return json.RawMessage(raw), nil
}
// SetRelatedLookup installs the producer identity and resolver closure the
// ObservationContext will use to answer GetRelated during rule evaluation.
// A nil lookup disables GetRelated (returns an empty slice).
func (oc *ObservationContext) SetRelatedLookup(producerCheckerID string, lookup RelatedObservationLookup) {
oc.mu.Lock()
defer oc.mu.Unlock()
oc.producerCheckerID = producerCheckerID
oc.relatedLookup = lookup
}
// GetRelated returns observations produced by other checkers on DiscoveryEntry
// records this target's producer has published. When no discovery storage is
// wired (or this checker never published any entries), it returns an empty
// slice — callers must tolerate that per SDK contract.
func (oc *ObservationContext) GetRelated(ctx context.Context, key happydns.ObservationKey) ([]happydns.RelatedObservation, error) {
oc.mu.Lock()
producer := oc.producerCheckerID
lookup := oc.relatedLookup
oc.mu.Unlock()
if lookup == nil || producer == "" {
return nil, nil
}
return lookup(ctx, producer, oc.target, key)
}
// Data returns all cached observation data as pre-serialized JSON.
func (oc *ObservationContext) Data() map[happydns.ObservationKey]json.RawMessage {
oc.mu.Lock()
@ -241,19 +302,19 @@ func HasHTMLReporter() bool {
return htmlReporterCached
}
// GetHTMLReport renders an HTML report for the given observation key and raw JSON data.
// GetHTMLReport renders an HTML report for the given observation key.
// Returns (html, true, nil) if the provider supports HTML reports, or ("", false, nil) if not.
func GetHTMLReport(key happydns.ObservationKey, raw json.RawMessage) (string, bool, error) {
return getHTMLReport(sdk.FindObservationProvider(key), key, raw)
func GetHTMLReport(key happydns.ObservationKey, rc happydns.ReportContext) (string, bool, error) {
return getHTMLReport(sdk.FindObservationProvider(key), key, rc)
}
// GetHTMLReportCtx is like GetHTMLReport but resolves the provider through
// the ObservationContext, respecting per-context overrides.
func (oc *ObservationContext) GetHTMLReportCtx(key happydns.ObservationKey, raw json.RawMessage) (string, bool, error) {
return getHTMLReport(oc.getProvider(key), key, raw)
func (oc *ObservationContext) GetHTMLReportCtx(key happydns.ObservationKey, rc happydns.ReportContext) (string, bool, error) {
return getHTMLReport(oc.getProvider(key), key, rc)
}
func getHTMLReport(provider happydns.ObservationProvider, key happydns.ObservationKey, raw json.RawMessage) (string, bool, error) {
func getHTMLReport(provider happydns.ObservationProvider, key happydns.ObservationKey, rc happydns.ReportContext) (string, bool, error) {
if provider == nil {
return "", false, fmt.Errorf("no observation provider registered for key %q", key)
}
@ -262,7 +323,7 @@ func getHTMLReport(provider happydns.ObservationProvider, key happydns.Observati
if !ok {
return "", false, nil
}
html, err := hr.GetHTMLReport(raw)
html, err := hr.GetHTMLReport(rc)
return html, true, err
}
@ -279,19 +340,19 @@ func HasMetricsReporter() bool {
return metricsReporterCached
}
// GetMetrics extracts metrics for the given observation key and raw JSON data.
// GetMetrics extracts metrics for the given observation key.
// Returns (metrics, true, nil) if the provider supports metrics, or (nil, false, nil) if not.
func GetMetrics(key happydns.ObservationKey, raw json.RawMessage, collectedAt time.Time) ([]happydns.CheckMetric, bool, error) {
return getMetrics(sdk.FindObservationProvider(key), key, raw, collectedAt)
func GetMetrics(key happydns.ObservationKey, rc happydns.ReportContext, collectedAt time.Time) ([]happydns.CheckMetric, bool, error) {
return getMetrics(sdk.FindObservationProvider(key), key, rc, collectedAt)
}
// GetMetricsCtx is like GetMetrics but resolves the provider through
// the ObservationContext, respecting per-context overrides.
func (oc *ObservationContext) GetMetricsCtx(key happydns.ObservationKey, raw json.RawMessage, collectedAt time.Time) ([]happydns.CheckMetric, bool, error) {
return getMetrics(oc.getProvider(key), key, raw, collectedAt)
func (oc *ObservationContext) GetMetricsCtx(key happydns.ObservationKey, rc happydns.ReportContext, collectedAt time.Time) ([]happydns.CheckMetric, bool, error) {
return getMetrics(oc.getProvider(key), key, rc, collectedAt)
}
func getMetrics(provider happydns.ObservationProvider, key happydns.ObservationKey, raw json.RawMessage, collectedAt time.Time) ([]happydns.CheckMetric, bool, error) {
func getMetrics(provider happydns.ObservationProvider, key happydns.ObservationKey, rc happydns.ReportContext, collectedAt time.Time) ([]happydns.CheckMetric, bool, error) {
if provider == nil {
return nil, false, fmt.Errorf("no observation provider registered for key %q", key)
}
@ -300,16 +361,77 @@ func getMetrics(provider happydns.ObservationProvider, key happydns.ObservationK
if !ok {
return nil, false, nil
}
metrics, err := mr.ExtractMetrics(raw, collectedAt)
metrics, err := mr.ExtractMetrics(rc, collectedAt)
return metrics, true, err
}
// BuildReportContext returns a ReportContext backed by the primary payload
// and a lazy Related(key) resolver that delegates to the installed lookup.
// When lookup is nil (no discovery storage), the returned context has no
// related observations and behaves like sdk.StaticReportContext(raw).
func BuildReportContext(ctx context.Context, producerCheckerID string, target happydns.CheckTarget, raw json.RawMessage, lookup RelatedObservationLookup) happydns.ReportContext {
if lookup == nil || producerCheckerID == "" {
return sdk.StaticReportContext(raw)
}
return &lazyReportContext{
ctx: ctx,
data: raw,
lookup: lookup,
producer: producerCheckerID,
target: target,
cache: make(map[happydns.ObservationKey][]happydns.RelatedObservation),
}
}
// lazyReportContext resolves Related(key) on first access against a host-side lookup closure.
type lazyReportContext struct {
mu sync.Mutex
ctx context.Context
data json.RawMessage
lookup RelatedObservationLookup
producer string
target happydns.CheckTarget
cache map[happydns.ObservationKey][]happydns.RelatedObservation
}
func (l *lazyReportContext) Data() json.RawMessage { return l.data }
func (l *lazyReportContext) Related(key happydns.ObservationKey) []happydns.RelatedObservation {
l.mu.Lock()
if cached, ok := l.cache[key]; ok {
l.mu.Unlock()
return cached
}
l.mu.Unlock()
out, err := l.lookup(l.ctx, l.producer, l.target, key)
if err != nil {
log.Printf("lazyReportContext: Related(%q): %v", key, err)
return nil
}
l.mu.Lock()
l.cache[key] = out
l.mu.Unlock()
return out
}
// GetHTMLReportWithContext renders an HTML report using a pre-built ReportContext
// (which may carry Related observations). Returns (html, true, nil) when supported.
func GetHTMLReportWithContext(key happydns.ObservationKey, rc happydns.ReportContext) (string, bool, error) {
return getHTMLReport(sdk.FindObservationProvider(key), key, rc)
}
// GetMetricsWithContext extracts metrics using a pre-built ReportContext.
func GetMetricsWithContext(key happydns.ObservationKey, rc happydns.ReportContext, collectedAt time.Time) ([]happydns.CheckMetric, bool, error) {
return getMetrics(sdk.FindObservationProvider(key), key, rc, collectedAt)
}
// GetAllMetrics extracts metrics from all observation keys in a snapshot.
func GetAllMetrics(snap *happydns.ObservationSnapshot) ([]happydns.CheckMetric, error) {
var allMetrics []happydns.CheckMetric
var errs []error
for key, raw := range snap.Data {
metrics, supported, err := GetMetrics(key, raw, snap.CollectedAt)
metrics, supported, err := GetMetrics(key, sdk.StaticReportContext(raw), snap.CollectedAt)
if err != nil {
errs = append(errs, fmt.Errorf("observation %q: %w", key, err))
continue

View file

@ -53,6 +53,8 @@ const maxResponseBodySize = 10 << 20 // 10 MiB
type HTTPObservationProvider struct {
observationKey happydns.ObservationKey
endpoint string // base URL without trailing slash
lastEntries []happydns.DiscoveryEntry // entries from the last Collect response, surfaced via DiscoverEntries
}
// NewHTTPObservationProvider creates a new HTTP-backed observation provider.
@ -114,7 +116,85 @@ func (p *HTTPObservationProvider) Collect(ctx context.Context, opts happydns.Che
return nil, fmt.Errorf("HTTP provider %s: remote returned empty data", p.observationKey)
}
p.lastEntries = result.Entries
// Return json.RawMessage directly - it implements json.Marshaler,
// so ObservationContext.Get() won't double-encode it.
return result.Data, nil
}
// DiscoverEntries implements sdk.DiscoveryPublisher: it exposes the entries
// carried in the last /collect response so the engine can ingest them
// through the same path as in-process providers.
func (p *HTTPObservationProvider) DiscoverEntries(_ any) ([]happydns.DiscoveryEntry, error) {
return p.lastEntries, nil
}
// report posts an ExternalReportRequest to the remote /report endpoint and
// returns the raw response body. The related map is built from the
// ReportContext's Related(key) for the caller-supplied keys so the remote
// reporter can consume cross-checker observations without an extra lookup.
func (p *HTTPObservationProvider) report(ctx context.Context, rc happydns.ReportContext, keys []happydns.ObservationKey) ([]byte, error) {
related := make(map[happydns.ObservationKey][]happydns.RelatedObservation, len(keys))
for _, k := range keys {
if rs := rc.Related(k); len(rs) > 0 {
related[k] = rs
}
}
reqBody := happydns.ExternalReportRequest{
Key: p.observationKey,
Data: rc.Data(),
Related: related,
}
body, err := json.Marshal(reqBody)
if err != nil {
return nil, fmt.Errorf("HTTP provider %s: marshal report request: %w", p.observationKey, err)
}
url := p.endpoint + "/report"
httpReq, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body))
if err != nil {
return nil, fmt.Errorf("HTTP provider %s: create report request: %w", p.observationKey, err)
}
httpReq.Header.Set("Content-Type", "application/json")
resp, err := httpClient.Do(httpReq)
if err != nil {
return nil, fmt.Errorf("HTTP provider %s: report request failed: %w", p.observationKey, err)
}
defer resp.Body.Close()
if resp.StatusCode == http.StatusNotImplemented {
return nil, fmt.Errorf("HTTP provider %s: remote does not support /report", p.observationKey)
}
if resp.StatusCode != http.StatusOK {
respBody, _ := io.ReadAll(io.LimitReader(resp.Body, maxErrorBodySize))
return nil, fmt.Errorf("HTTP provider %s: report returned status %d: %s", p.observationKey, resp.StatusCode, string(respBody))
}
return io.ReadAll(io.LimitReader(resp.Body, maxResponseBodySize))
}
// GetHTMLReport implements happydns.CheckerHTMLReporter by forwarding to
// POST /report. Related observations present in rc are forwarded under the
// provider's own observation key — the only key that can meaningfully be
// consumed by the remote reporter.
func (p *HTTPObservationProvider) GetHTMLReport(rc happydns.ReportContext) (string, error) {
body, err := p.report(context.Background(), rc, []happydns.ObservationKey{p.observationKey})
if err != nil {
return "", err
}
return string(body), nil
}
// ExtractMetrics implements happydns.CheckerMetricsReporter by forwarding to
// POST /report and expecting a JSON array of happydns.CheckMetric.
func (p *HTTPObservationProvider) ExtractMetrics(rc happydns.ReportContext, _ time.Time) ([]happydns.CheckMetric, error) {
body, err := p.report(context.Background(), rc, []happydns.ObservationKey{p.observationKey})
if err != nil {
return nil, err
}
var metrics []happydns.CheckMetric
if err := json.Unmarshal(body, &metrics); err != nil {
return nil, fmt.Errorf("HTTP provider %s: decode metrics response: %w", p.observationKey, err)
}
return metrics, nil
}

View file

@ -29,7 +29,9 @@ import (
"net/http/httptest"
"strings"
"testing"
"time"
sdk "git.happydns.org/checker-sdk-go/checker"
"git.happydns.org/happyDomain/model"
)
@ -206,6 +208,82 @@ func TestHTTPObservationProvider_CollectConnectionRefused(t *testing.T) {
}
}
func TestHTTPObservationProvider_CollectForwardsEntries(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(happydns.ExternalCollectResponse{
Data: json.RawMessage(`{"ok":true}`),
Entries: []happydns.DiscoveryEntry{
{Type: "tls.endpoint.v1", Ref: "a.example.com:25"},
{Type: "tls.endpoint.v1", Ref: "a.example.com:465"},
},
})
}))
defer srv.Close()
p := NewHTTPObservationProvider("k", srv.URL)
if _, err := p.Collect(context.Background(), nil); err != nil {
t.Fatalf("Collect: %v", err)
}
entries, err := p.DiscoverEntries(nil)
if err != nil {
t.Fatalf("DiscoverEntries: %v", err)
}
if len(entries) != 2 || entries[1].Ref != "a.example.com:465" {
t.Fatalf("unexpected entries: %+v", entries)
}
}
func TestHTTPObservationProvider_GetHTMLReportForwardsRelated(t *testing.T) {
var gotReq happydns.ExternalReportRequest
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/report" {
t.Errorf("path = %q, want /report", r.URL.Path)
}
if err := json.NewDecoder(r.Body).Decode(&gotReq); err != nil {
t.Fatalf("decode: %v", err)
}
w.Header().Set("Content-Type", "text/html")
io.WriteString(w, "<html>ok</html>")
}))
defer srv.Close()
p := NewHTTPObservationProvider("tls", srv.URL)
related := map[happydns.ObservationKey][]happydns.RelatedObservation{
"tls": {
{CheckerID: "xmpp", Key: "tls", Data: json.RawMessage(`{"v":1}`), Ref: "host:443", CollectedAt: time.Unix(42, 0).UTC()},
},
}
rc := sdk.NewReportContext(json.RawMessage(`{"primary":true}`), related)
html, err := p.GetHTMLReport(rc)
if err != nil {
t.Fatalf("GetHTMLReport: %v", err)
}
if html != "<html>ok</html>" {
t.Fatalf("html = %q", html)
}
if gotReq.Key != "tls" {
t.Errorf("Key = %q, want tls", gotReq.Key)
}
if len(gotReq.Related["tls"]) != 1 || gotReq.Related["tls"][0].Ref != "host:443" {
t.Errorf("Related not forwarded: %+v", gotReq.Related)
}
}
func TestHTTPObservationProvider_GetHTMLReportSurfaces501(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "not implemented", http.StatusNotImplemented)
}))
defer srv.Close()
p := NewHTTPObservationProvider("k", srv.URL)
_, err := p.GetHTMLReport(sdk.StaticReportContext(json.RawMessage(`{}`)))
if err == nil || !strings.Contains(err.Error(), "does not support") {
t.Fatalf("want 'does not support' error, got: %v", err)
}
}
func TestHTTPObservationProvider_IntegrationWithObservationContext(t *testing.T) {
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")

View file

@ -45,6 +45,8 @@ type Storage interface {
checker.CheckerOptionsStorage
checker.CheckEvaluationStorage
checker.ExecutionStorage
checker.DiscoveryEntryStorage
checker.DiscoveryObservationStorage
checker.ObservationCacheStorage
checker.ObservationSnapshotStorage
checker.SchedulerStateStorage

View file

@ -0,0 +1,244 @@
// This file is part of the happyDomain (R) project.
// Copyright (c) 2020-2026 happyDomain
// Authors: Pierre-Olivier Mercier, et al.
//
// This program is offered under a commercial and under the AGPL license.
// For commercial licensing, contact us at <contact@happydomain.org>.
//
// For AGPL licensing:
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
// discovery.go persists the two host-side indexes behind the cross-checker
// discovery mechanism described in docs/checker-discovery.md:
//
// - dscent|{producer}|{target}|{type}|{ref} primary record
// - dscent-tgt|{target}|{producer}|{type}|{ref} target lookup (auto-fill)
// - dscobs|{producer}|{target}|{ref}|{consumer}|{k} observation lineage
// - dscobs-snap|{snapshotId}|... cascade on snapshot delete
//
// Refs and observation keys are opaque to the host; we trust producers not
// to embed "|" in them (the SDK doc recommends short, deterministic values
// such as "host:port" or a sha1 digest).
package database
import (
"fmt"
"log"
"strings"
"git.happydns.org/happyDomain/model"
)
func dscEntryKey(producerID string, target happydns.CheckTarget, typ, ref string) string {
return fmt.Sprintf("dscent|%s|%s|%s|%s", producerID, target.String(), typ, ref)
}
func dscEntryTargetIndexKey(producerID string, target happydns.CheckTarget, typ, ref string) string {
return fmt.Sprintf("dscent-tgt|%s|%s|%s|%s", target.String(), producerID, typ, ref)
}
func dscObsKey(producerID string, target happydns.CheckTarget, ref, consumerID string, obsKey happydns.ObservationKey) string {
return fmt.Sprintf("dscobs|%s|%s|%s|%s|%s", producerID, target.String(), ref, consumerID, obsKey)
}
func dscObsSnapIndexKey(snapshotID happydns.Identifier, primary string) string {
// The primary key is appended verbatim so cascade delete can recover it
// without parsing the suffix; the value carries it too for safety.
return fmt.Sprintf("dscobs-snap|%s|%s", snapshotID.String(), primary)
}
// --- DiscoveryEntry storage -------------------------------------------------
func (s *KVStorage) ListDiscoveryEntriesByTarget(target happydns.CheckTarget) ([]*happydns.StoredDiscoveryEntry, error) {
prefix := fmt.Sprintf("dscent-tgt|%s|", target.String())
iter := s.db.Search(prefix)
defer iter.Release()
var out []*happydns.StoredDiscoveryEntry
for iter.Next() {
rest := strings.TrimPrefix(iter.Key(), prefix)
parts := strings.SplitN(rest, "|", 3)
if len(parts) != 3 {
continue
}
entry := &happydns.StoredDiscoveryEntry{}
if err := s.db.Get(dscEntryKey(parts[0], target, parts[1], parts[2]), entry); err != nil {
// Stale index entry — ignore; tidy will eventually clean it.
continue
}
out = append(out, entry)
}
return out, nil
}
func (s *KVStorage) ListDiscoveryEntriesByProducer(producerID string, target happydns.CheckTarget) ([]*happydns.StoredDiscoveryEntry, error) {
prefix := fmt.Sprintf("dscent|%s|%s|", producerID, target.String())
iter := s.db.Search(prefix)
defer iter.Release()
var out []*happydns.StoredDiscoveryEntry
for iter.Next() {
entry := &happydns.StoredDiscoveryEntry{}
if err := s.db.DecodeData(iter.Value(), entry); err != nil {
continue
}
out = append(out, entry)
}
return out, nil
}
func (s *KVStorage) ListAllDiscoveryEntries() (happydns.Iterator[happydns.StoredDiscoveryEntry], error) {
iter := s.db.Search("dscent|")
return NewKVIterator[happydns.StoredDiscoveryEntry](s.db, iter), nil
}
// ReplaceDiscoveryEntries atomically replaces the set of entries stored for
// (producerID, target): everything previously stored is deleted, then the
// new set is written. Passing an empty `entries` slice simply clears.
func (s *KVStorage) ReplaceDiscoveryEntries(producerID string, target happydns.CheckTarget, entries []happydns.DiscoveryEntry) error {
if err := s.DeleteDiscoveryEntriesByProducer(producerID, target); err != nil {
return err
}
for _, e := range entries {
stored := &happydns.StoredDiscoveryEntry{
ProducerID: producerID,
Target: target,
Type: e.Type,
Ref: e.Ref,
Payload: e.Payload,
}
if err := s.db.Put(dscEntryKey(producerID, target, e.Type, e.Ref), stored); err != nil {
return err
}
if err := s.db.Put(dscEntryTargetIndexKey(producerID, target, e.Type, e.Ref), true); err != nil {
return err
}
}
return nil
}
// RestoreDiscoveryEntry writes an entry at its canonical key and rebuilds
// its target index. Used by the backup restore path.
func (s *KVStorage) RestoreDiscoveryEntry(entry *happydns.StoredDiscoveryEntry) error {
if err := s.db.Put(dscEntryKey(entry.ProducerID, entry.Target, entry.Type, entry.Ref), entry); err != nil {
return err
}
return s.db.Put(dscEntryTargetIndexKey(entry.ProducerID, entry.Target, entry.Type, entry.Ref), true)
}
func (s *KVStorage) DeleteDiscoveryEntriesByProducer(producerID string, target happydns.CheckTarget) error {
prefix := fmt.Sprintf("dscent|%s|%s|", producerID, target.String())
iter := s.db.Search(prefix)
defer iter.Release()
for iter.Next() {
rest := strings.TrimPrefix(iter.Key(), prefix)
parts := strings.SplitN(rest, "|", 2)
if len(parts) == 2 {
if err := s.db.Delete(dscEntryTargetIndexKey(producerID, target, parts[0], parts[1])); err != nil {
log.Printf("DeleteDiscoveryEntriesByProducer: failed to delete target index: %v", err)
}
}
if err := s.db.Delete(iter.Key()); err != nil {
return err
}
}
return nil
}
func (s *KVStorage) ClearDiscoveryEntries() error {
if err := s.clearByPrefix("dscent-tgt|"); err != nil {
return err
}
return s.clearByPrefix("dscent|")
}
// --- DiscoveryObservationRef storage ----------------------------------------
func (s *KVStorage) PutDiscoveryObservationRef(ref *happydns.DiscoveryObservationRef) error {
primary := dscObsKey(ref.ProducerID, ref.Target, ref.Ref, ref.ConsumerID, ref.Key)
// If a previous ref exists at the same primary key under a different
// snapshot, clean up its stale snap-index so a later cascade delete for
// that earlier snapshot doesn't wipe the primary this call just wrote.
old := &happydns.DiscoveryObservationRef{}
if err := s.db.Get(primary, old); err == nil && !old.SnapshotID.Equals(ref.SnapshotID) {
_ = s.db.Delete(dscObsSnapIndexKey(old.SnapshotID, primary))
}
if err := s.db.Put(primary, ref); err != nil {
return err
}
return s.db.Put(dscObsSnapIndexKey(ref.SnapshotID, primary), primary)
}
func (s *KVStorage) ListDiscoveryObservationRefs(producerID string, target happydns.CheckTarget, ref string) ([]*happydns.DiscoveryObservationRef, error) {
prefix := fmt.Sprintf("dscobs|%s|%s|%s|", producerID, target.String(), ref)
iter := s.db.Search(prefix)
defer iter.Release()
var out []*happydns.DiscoveryObservationRef
for iter.Next() {
r := &happydns.DiscoveryObservationRef{}
if err := s.db.DecodeData(iter.Value(), r); err != nil {
continue
}
out = append(out, r)
}
return out, nil
}
func (s *KVStorage) ListAllDiscoveryObservationRefs() (happydns.Iterator[happydns.DiscoveryObservationRef], error) {
iter := s.db.Search("dscobs|")
return NewKVIterator[happydns.DiscoveryObservationRef](s.db, iter), nil
}
// RestoreDiscoveryObservationRef writes a ref at its canonical key and
// rebuilds its snapshot index. Used by the backup restore path.
func (s *KVStorage) RestoreDiscoveryObservationRef(ref *happydns.DiscoveryObservationRef) error {
primary := dscObsKey(ref.ProducerID, ref.Target, ref.Ref, ref.ConsumerID, ref.Key)
if err := s.db.Put(primary, ref); err != nil {
return err
}
return s.db.Put(dscObsSnapIndexKey(ref.SnapshotID, primary), primary)
}
func (s *KVStorage) DeleteDiscoveryObservationRefsForSnapshot(snapshotID happydns.Identifier) error {
prefix := fmt.Sprintf("dscobs-snap|%s|", snapshotID.String())
iter := s.db.Search(prefix)
defer iter.Release()
for iter.Next() {
var primary string
if err := s.db.DecodeData(iter.Value(), &primary); err != nil || primary == "" {
// Fall back to extracting from the key suffix.
primary = strings.TrimPrefix(iter.Key(), prefix)
}
if err := s.db.Delete(primary); err != nil {
log.Printf("DeleteDiscoveryObservationRefsForSnapshot: failed to delete primary %s: %v", primary, err)
}
if err := s.db.Delete(iter.Key()); err != nil {
return err
}
}
return nil
}
func (s *KVStorage) ClearDiscoveryObservationRefs() error {
if err := s.clearByPrefix("dscobs-snap|"); err != nil {
return err
}
return s.clearByPrefix("dscobs|")
}

View file

@ -0,0 +1,240 @@
// This file is part of the happyDomain (R) project.
// Copyright (c) 2020-2026 happyDomain
// Authors: Pierre-Olivier Mercier, et al.
//
// This program is offered under a commercial and under the AGPL license.
// For commercial licensing, contact us at <contact@happydomain.org>.
//
// For AGPL licensing:
// This program is free software: you can redistribute it and/or modify
// it under the terms of the GNU Affero General Public License as published by
// the Free Software Foundation, either version 3 of the License, or
// (at your option) any later version.
//
// This program is distributed in the hope that it will be useful,
// but WITHOUT ANY WARRANTY; without even the implied warranty of
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
// GNU Affero General Public License for more details.
//
// You should have received a copy of the GNU Affero General Public License
// along with this program. If not, see <https://www.gnu.org/licenses/>.
package database
import (
"encoding/json"
"fmt"
"sort"
"strings"
"testing"
"time"
"git.happydns.org/happyDomain/internal/storage"
"git.happydns.org/happyDomain/model"
)
// fakeKV is a minimal in-memory KV used only by these tests — inmemory package
// imports kvtpl, so we cannot import it here.
type fakeKV struct {
data map[string]json.RawMessage
}
func newFakeKV() *fakeKV { return &fakeKV{data: map[string]json.RawMessage{}} }
func (f *fakeKV) Close() error { return nil }
func (f *fakeKV) DecodeData(i any, v any) error {
b, ok := i.(json.RawMessage)
if !ok {
return fmt.Errorf("not a RawMessage (%T)", i)
}
return json.Unmarshal(b, v)
}
func (f *fakeKV) Has(key string) (bool, error) {
_, ok := f.data[key]
return ok, nil
}
func (f *fakeKV) Get(key string, v any) error {
raw, ok := f.data[key]
if !ok {
return happydns.ErrNotFound
}
return json.Unmarshal(raw, v)
}
func (f *fakeKV) Put(key string, v any) error {
b, err := json.Marshal(v)
if err != nil {
return err
}
f.data[key] = b
return nil
}
func (f *fakeKV) FindIdentifierKey(prefix string) (string, happydns.Identifier, error) {
id, err := happydns.NewRandomIdentifier()
if err != nil {
return "", nil, err
}
return prefix + id.String(), id, nil
}
func (f *fakeKV) Delete(key string) error {
delete(f.data, key)
return nil
}
func (f *fakeKV) Search(prefix string) storage.Iterator {
keys := make([]string, 0)
for k := range f.data {
if strings.HasPrefix(k, prefix) {
keys = append(keys, k)
}
}
sort.Strings(keys)
return &fakeIter{data: f.data, keys: keys, idx: -1}
}
type fakeIter struct {
data map[string]json.RawMessage
keys []string
idx int
}
func (i *fakeIter) Release() {}
func (i *fakeIter) Next() bool { i.idx++; return i.idx < len(i.keys) }
func (i *fakeIter) Valid() bool { return i.idx >= 0 && i.idx < len(i.keys) }
func (i *fakeIter) Key() string { return i.keys[i.idx] }
func (i *fakeIter) Value() any { return i.data[i.keys[i.idx]] }
func (i *fakeIter) Err() error { return nil }
func newDiscoveryTestStore() *KVStorage {
return &KVStorage{db: newFakeKV()}
}
func TestReplaceDiscoveryEntriesRoundTrip(t *testing.T) {
s := newDiscoveryTestStore()
target := happydns.CheckTarget{DomainId: "domA"}
entries := []happydns.DiscoveryEntry{
{Type: "tls.endpoint.v1", Ref: "host:443", Payload: json.RawMessage(`{"host":"a"}`)},
{Type: "tls.endpoint.v1", Ref: "host:465", Payload: json.RawMessage(`{"host":"b"}`)},
}
if err := s.ReplaceDiscoveryEntries("checker-srv", target, entries); err != nil {
t.Fatalf("ReplaceDiscoveryEntries: %v", err)
}
byProducer, err := s.ListDiscoveryEntriesByProducer("checker-srv", target)
if err != nil {
t.Fatalf("ListDiscoveryEntriesByProducer: %v", err)
}
if len(byProducer) != 2 {
t.Fatalf("want 2 entries, got %d", len(byProducer))
}
byTarget, err := s.ListDiscoveryEntriesByTarget(target)
if err != nil {
t.Fatalf("ListDiscoveryEntriesByTarget: %v", err)
}
if len(byTarget) != 2 {
t.Fatalf("want 2 entries via target index, got %d", len(byTarget))
}
}
func TestReplaceDiscoveryEntriesReplacesAtomically(t *testing.T) {
s := newDiscoveryTestStore()
target := happydns.CheckTarget{DomainId: "domA"}
if err := s.ReplaceDiscoveryEntries("p", target, []happydns.DiscoveryEntry{
{Type: "t", Ref: "r1"},
{Type: "t", Ref: "r2"},
}); err != nil {
t.Fatal(err)
}
if err := s.ReplaceDiscoveryEntries("p", target, []happydns.DiscoveryEntry{
{Type: "t", Ref: "r3"},
}); err != nil {
t.Fatal(err)
}
got, _ := s.ListDiscoveryEntriesByProducer("p", target)
if len(got) != 1 || got[0].Ref != "r3" {
t.Fatalf("replace did not clear previous set: %#v", got)
}
viaTarget, _ := s.ListDiscoveryEntriesByTarget(target)
if len(viaTarget) != 1 || viaTarget[0].Ref != "r3" {
t.Fatalf("target index diverged from primary: %#v", viaTarget)
}
}
func TestListDiscoveryEntriesByTargetAggregatesProducers(t *testing.T) {
s := newDiscoveryTestStore()
target := happydns.CheckTarget{DomainId: "domA"}
if err := s.ReplaceDiscoveryEntries("p1", target, []happydns.DiscoveryEntry{{Type: "t", Ref: "a"}}); err != nil {
t.Fatal(err)
}
if err := s.ReplaceDiscoveryEntries("p2", target, []happydns.DiscoveryEntry{{Type: "t", Ref: "b"}}); err != nil {
t.Fatal(err)
}
got, _ := s.ListDiscoveryEntriesByTarget(target)
if len(got) != 2 {
t.Fatalf("want 2 entries from two producers, got %d", len(got))
}
}
func TestDiscoveryObservationRefCascadeOnSnapshotDelete(t *testing.T) {
s := newDiscoveryTestStore()
target := happydns.CheckTarget{DomainId: "domA"}
snap := happydns.Identifier{1, 2, 3}
other := happydns.Identifier{4, 5, 6}
refs := []*happydns.DiscoveryObservationRef{
{ProducerID: "p", Target: target, Ref: "r1", ConsumerID: "c", Key: "k", SnapshotID: snap, CollectedAt: time.Now()},
{ProducerID: "p", Target: target, Ref: "r2", ConsumerID: "c", Key: "k", SnapshotID: snap, CollectedAt: time.Now()},
{ProducerID: "p", Target: target, Ref: "r1", ConsumerID: "c", Key: "k", SnapshotID: other, CollectedAt: time.Now()},
}
for _, r := range refs {
if err := s.PutDiscoveryObservationRef(r); err != nil {
t.Fatal(err)
}
}
if err := s.DeleteDiscoveryObservationRefsForSnapshot(snap); err != nil {
t.Fatalf("cascade delete: %v", err)
}
remaining, _ := s.ListDiscoveryObservationRefs("p", target, "r1")
if len(remaining) != 1 || !remaining[0].SnapshotID.Equals(other) {
t.Fatalf("cascade delete left stale data: %#v", remaining)
}
remaining2, _ := s.ListDiscoveryObservationRefs("p", target, "r2")
if len(remaining2) != 0 {
t.Fatalf("cascade delete missed snapshot refs: %#v", remaining2)
}
}
func TestPutDiscoveryObservationRefUpsert(t *testing.T) {
s := newDiscoveryTestStore()
target := happydns.CheckTarget{DomainId: "domA"}
first := &happydns.DiscoveryObservationRef{
ProducerID: "p", Target: target, Ref: "r", ConsumerID: "c", Key: "k",
SnapshotID: happydns.Identifier{1}, CollectedAt: time.Now().Add(-time.Hour),
}
second := &happydns.DiscoveryObservationRef{
ProducerID: "p", Target: target, Ref: "r", ConsumerID: "c", Key: "k",
SnapshotID: happydns.Identifier{2}, CollectedAt: time.Now(),
}
if err := s.PutDiscoveryObservationRef(first); err != nil {
t.Fatal(err)
}
if err := s.PutDiscoveryObservationRef(second); err != nil {
t.Fatal(err)
}
got, _ := s.ListDiscoveryObservationRefs("p", target, "r")
if len(got) != 1 {
t.Fatalf("upsert should keep a single ref per tuple, got %d", len(got))
}
if !got[0].SnapshotID.Equals(second.SnapshotID) {
t.Fatalf("latest ref should win, got SnapshotID=%v", got[0].SnapshotID)
}
}

View file

@ -34,27 +34,37 @@ import (
// checkerEngine implements the happydns.CheckerEngine interface.
type checkerEngine struct {
optionsUC *CheckerOptionsUsecase
evalStore CheckEvaluationStorage
execStore ExecutionStorage
snapStore ObservationSnapshotStorage
cacheStore ObservationCacheStorage
optionsUC *CheckerOptionsUsecase
evalStore CheckEvaluationStorage
execStore ExecutionStorage
snapStore ObservationSnapshotStorage
cacheStore ObservationCacheStorage
entryStore DiscoveryEntryStorage
obsRefStore DiscoveryObservationStorage
relatedLookup checkerPkg.RelatedObservationLookup
}
// NewCheckerEngine creates a new CheckerEngine implementation.
// NewCheckerEngine creates a new CheckerEngine implementation. Passing nil
// for entryStore/obsRefStore disables cross-checker discovery; the engine
// then behaves exactly as before the discovery mechanism was introduced.
func NewCheckerEngine(
optionsUC *CheckerOptionsUsecase,
evalStore CheckEvaluationStorage,
execStore ExecutionStorage,
snapStore ObservationSnapshotStorage,
cacheStore ObservationCacheStorage,
entryStore DiscoveryEntryStorage,
obsRefStore DiscoveryObservationStorage,
) happydns.CheckerEngine {
return &checkerEngine{
optionsUC: optionsUC,
evalStore: evalStore,
execStore: execStore,
snapStore: snapStore,
cacheStore: cacheStore,
optionsUC: optionsUC,
evalStore: evalStore,
execStore: execStore,
snapStore: snapStore,
cacheStore: cacheStore,
entryStore: entryStore,
obsRefStore: obsRefStore,
relatedLookup: newRelatedLookup(entryStore, obsRefStore, snapStore),
}
}
@ -140,7 +150,7 @@ func (e *checkerEngine) RunExecution(ctx context.Context, exec *happydns.Executi
func (e *checkerEngine) runPipeline(ctx context.Context, def *happydns.CheckerDefinition, target happydns.CheckTarget, plan *happydns.CheckPlan, planID *happydns.Identifier, runOpts happydns.CheckerOptions) (happydns.CheckState, *happydns.CheckEvaluation, error) {
// Resolve options (stored + run + auto-fill).
mergedOpts, err := e.optionsUC.BuildMergedCheckerOptionsWithAutoFill(def.ID, happydns.TargetIdentifier(target.UserId), happydns.TargetIdentifier(target.DomainId), happydns.TargetIdentifier(target.ServiceId), runOpts)
mergedOpts, injectedEntries, err := e.optionsUC.BuildMergedCheckerOptionsWithAutoFill(def.ID, happydns.TargetIdentifier(target.UserId), happydns.TargetIdentifier(target.DomainId), happydns.TargetIdentifier(target.ServiceId), runOpts)
if err != nil {
return happydns.CheckState{}, nil, fmt.Errorf("resolving options: %w", err)
}
@ -175,6 +185,10 @@ func (e *checkerEngine) runPipeline(ctx context.Context, def *happydns.CheckerDe
// Create observation context for lazy data collection.
obsCtx := checkerPkg.NewObservationContext(target, mergedOpts, cacheLookup, freshness)
if e.relatedLookup != nil {
obsCtx.SetRelatedLookup(def.ID, e.relatedLookup)
}
// If an endpoint is configured, override observation providers with HTTP transport.
if endpoint, ok := mergedOpts["endpoint"].(string); ok && endpoint != "" {
for _, key := range def.ObservationKeys {
@ -188,11 +202,17 @@ func (e *checkerEngine) runPipeline(ctx context.Context, def *happydns.CheckerDe
if plan != nil && !plan.IsRuleEnabled(rule.Name()) {
continue
}
state := rule.Evaluate(ctx, obsCtx, mergedOpts)
if state.Code == "" {
state.Code = rule.Name()
ruleStates := rule.Evaluate(ctx, obsCtx, mergedOpts)
if len(ruleStates) == 0 {
ruleStates = []happydns.CheckState{{
Status: happydns.StatusUnknown,
Message: "rule returned no state",
}}
}
states = append(states, state)
for i := range ruleStates {
ruleStates[i].RuleName = rule.Name()
}
states = append(states, ruleStates...)
}
// Aggregate results.
@ -224,6 +244,43 @@ func (e *checkerEngine) runPipeline(ctx context.Context, def *happydns.CheckerDe
}
}
// Publish DiscoveryEntry records harvested during collection, always
// replacing the previous set for (producer, target) — including with
// an empty slice, so previously-published entries vanish when the run
// produces none.
if e.entryStore != nil {
var published []happydns.DiscoveryEntry
for _, list := range obsCtx.Entries() {
published = append(published, list...)
}
if err := e.entryStore.ReplaceDiscoveryEntries(def.ID, target, published); err != nil {
log.Printf("warning: failed to replace discovery entries for %s on %s: %v", def.ID, target.String(), err)
}
}
// Persist the consumer→entry lineage: for each entry that was fed into
// this run via AutoFillDiscoveryEntries, link every observation we just
// stored to the original producer's (producer, target, ref) tuple. A
// later GetRelated call from the producer walks these refs.
if e.obsRefStore != nil && len(injectedEntries) > 0 && len(snap.Data) > 0 {
for _, entry := range injectedEntries {
for key := range snap.Data {
ref := &happydns.DiscoveryObservationRef{
ProducerID: entry.ProducerID,
Target: entry.Target,
Ref: entry.Ref,
ConsumerID: def.ID,
Key: key,
SnapshotID: snap.Id,
CollectedAt: snap.CollectedAt,
}
if err := e.obsRefStore.PutDiscoveryObservationRef(ref); err != nil {
log.Printf("warning: failed to persist observation ref for %s/%s: %v", entry.ProducerID, entry.Ref, err)
}
}
}
}
// Persist evaluation.
eval := &happydns.CheckEvaluation{
PlanID: planID,
@ -239,3 +296,59 @@ func (e *checkerEngine) runPipeline(ctx context.Context, def *happydns.CheckerDe
return result, eval, nil
}
// RelatedLookup exposes the engine's Related resolver so controllers can
// build ReportContexts with cross-checker observations pre-resolved. Returns
// nil when discovery storage is not wired.
func (e *checkerEngine) RelatedLookup() checkerPkg.RelatedObservationLookup {
return e.relatedLookup
}
// newRelatedLookup builds the RelatedObservationLookup closure once at engine
// construction time. Returns nil when any required store is absent.
func newRelatedLookup(entryStore DiscoveryEntryStorage, obsRefStore DiscoveryObservationStorage, snapStore ObservationSnapshotStorage) checkerPkg.RelatedObservationLookup {
if entryStore == nil || obsRefStore == nil || snapStore == nil {
return nil
}
return func(_ context.Context, producerCheckerID string, target happydns.CheckTarget, key happydns.ObservationKey) ([]happydns.RelatedObservation, error) {
entries, err := entryStore.ListDiscoveryEntriesByProducer(producerCheckerID, target)
if err != nil {
return nil, err
}
var out []happydns.RelatedObservation
snapCache := make(map[string]*happydns.ObservationSnapshot)
for _, entry := range entries {
refs, err := obsRefStore.ListDiscoveryObservationRefs(producerCheckerID, target, entry.Ref)
if err != nil {
continue
}
for _, r := range refs {
if r.Key != key {
continue
}
snapID := r.SnapshotID.String()
snap, ok := snapCache[snapID]
if !ok {
snap, err = snapStore.GetSnapshot(r.SnapshotID)
if err != nil {
// Snapshot gone (TTL) — skip silently; implicit GC.
continue
}
snapCache[snapID] = snap
}
data, ok := snap.Data[r.Key]
if !ok {
continue
}
out = append(out, happydns.RelatedObservation{
CheckerID: r.ConsumerID,
Key: r.Key,
Data: data,
CollectedAt: r.CollectedAt,
Ref: r.Ref,
})
}
}
return out, nil
}
}

View file

@ -54,12 +54,12 @@ type testCheckRule struct {
func (r *testCheckRule) Name() string { return r.name }
func (r *testCheckRule) Description() string { return "test rule: " + r.name }
func (r *testCheckRule) Evaluate(ctx context.Context, obs happydns.ObservationGetter, opts happydns.CheckerOptions) happydns.CheckState {
func (r *testCheckRule) Evaluate(ctx context.Context, obs happydns.ObservationGetter, opts happydns.CheckerOptions) []happydns.CheckState {
var data map[string]any
if err := obs.Get(ctx, "test_obs", &data); err != nil {
return happydns.CheckState{Status: happydns.StatusError, Message: err.Error()}
return []happydns.CheckState{{Status: happydns.StatusError, Message: err.Error()}}
}
return happydns.CheckState{Status: r.status, Message: r.name + " passed", Code: r.name}
return []happydns.CheckState{{Status: r.status, Message: r.name + " passed"}}
}
func TestCheckerEngine_RunOK(t *testing.T) {
@ -82,7 +82,7 @@ func TestCheckerEngine_RunOK(t *testing.T) {
})
optionsUC := checkerUC.NewCheckerOptionsUsecase(store, nil)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store, store, store)
uid, _ := happydns.NewRandomIdentifier()
did, _ := happydns.NewRandomIdentifier()
@ -146,7 +146,7 @@ func TestCheckerEngine_RunWarn(t *testing.T) {
})
optionsUC := checkerUC.NewCheckerOptionsUsecase(store, nil)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store, store, store)
uid, _ := happydns.NewRandomIdentifier()
did, _ := happydns.NewRandomIdentifier()
@ -191,7 +191,7 @@ func TestCheckerEngine_RunPerRuleDisable(t *testing.T) {
})
optionsUC := checkerUC.NewCheckerOptionsUsecase(store, nil)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store, store, store)
uid, _ := happydns.NewRandomIdentifier()
did, _ := happydns.NewRandomIdentifier()
@ -225,8 +225,8 @@ func TestCheckerEngine_RunPerRuleDisable(t *testing.T) {
t.Errorf("expected status OK (only rule_a active), got %s", exec.Result.Status)
}
if eval.States[0].Code != "rule_a" {
t.Errorf("expected rule_a state, got code %s", eval.States[0].Code)
if eval.States[0].RuleName != "rule_a" {
t.Errorf("expected rule_a state, got rule %s", eval.States[0].RuleName)
}
}
@ -281,7 +281,7 @@ func TestCheckerEngine_RunNotFound(t *testing.T) {
t.Fatalf("Instantiate() returned error: %v", err)
}
optionsUC := checkerUC.NewCheckerOptionsUsecase(store, nil)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store, store, store)
uid, _ := happydns.NewRandomIdentifier()
target := happydns.CheckTarget{UserId: uid.String()}
@ -310,7 +310,7 @@ func TestCheckerEngine_RunWithScheduledTrigger(t *testing.T) {
})
optionsUC := checkerUC.NewCheckerOptionsUsecase(store, nil)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store, store, store)
uid, _ := happydns.NewRandomIdentifier()
did, _ := happydns.NewRandomIdentifier()
@ -363,7 +363,7 @@ func TestCheckerEngine_RunExecution_CheckerDisappeared(t *testing.T) {
})
optionsUC := checkerUC.NewCheckerOptionsUsecase(store, nil)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store, store, store)
uid, _ := happydns.NewRandomIdentifier()
target := happydns.CheckTarget{UserId: uid.String()}
@ -411,7 +411,7 @@ func TestCheckerEngine_RunPopulatesObservationCache(t *testing.T) {
})
optionsUC := checkerUC.NewCheckerOptionsUsecase(store, nil)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store, store, store)
uid, _ := happydns.NewRandomIdentifier()
did, _ := happydns.NewRandomIdentifier()
@ -494,7 +494,7 @@ func TestCheckerEngine_RunWithEndpointOverride(t *testing.T) {
}
optionsUC := checkerUC.NewCheckerOptionsUsecase(store, nil)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store, store, store)
uid, _ := happydns.NewRandomIdentifier()
did, _ := happydns.NewRandomIdentifier()
@ -559,7 +559,7 @@ func TestCheckerEngine_RunWithEndpointOverride_RemoteFailure(t *testing.T) {
}
optionsUC := checkerUC.NewCheckerOptionsUsecase(store, nil)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store, store, store)
uid, _ := happydns.NewRandomIdentifier()
did, _ := happydns.NewRandomIdentifier()
@ -584,3 +584,216 @@ func TestCheckerEngine_RunWithEndpointOverride_RemoteFailure(t *testing.T) {
t.Fatalf("expected 1 state, got %d", len(eval.States))
}
}
// discoveringProvider returns static data and publishes a deterministic
// DiscoveryEntry per run, simulating a producer checker.
type discoveringProvider struct {
key happydns.ObservationKey
entries []happydns.DiscoveryEntry
}
func (p *discoveringProvider) Key() happydns.ObservationKey { return p.key }
func (p *discoveringProvider) Collect(ctx context.Context, opts happydns.CheckerOptions) (any, error) {
return map[string]any{"ok": true}, nil
}
func (p *discoveringProvider) DiscoverEntries(_ any) ([]happydns.DiscoveryEntry, error) {
return p.entries, nil
}
func TestCheckerEngine_PublishesDiscoveryEntries(t *testing.T) {
store, err := inmemory.Instantiate()
if err != nil {
t.Fatalf("Instantiate() returned error: %v", err)
}
provider := &discoveringProvider{
key: "test_disc_obs",
entries: []happydns.DiscoveryEntry{
{Type: "tls.endpoint.v1", Ref: "mail.example.com:25", Payload: json.RawMessage(`{"port":25}`)},
{Type: "tls.endpoint.v1", Ref: "mail.example.com:465", Payload: json.RawMessage(`{"port":465}`)},
},
}
checker.RegisterObservationProvider(provider)
checker.RegisterChecker(&happydns.CheckerDefinition{
ID: "test_discovery_publisher",
Name: "Test Discovery Publisher",
Availability: happydns.CheckerAvailability{
ApplyToDomain: true,
},
Rules: []happydns.CheckRule{
&testCheckRuleReadingKey{name: "publish_rule", key: "test_disc_obs"},
},
})
optionsUC := checkerUC.NewCheckerOptionsUsecase(store, nil)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store, store, store)
uid, _ := happydns.NewRandomIdentifier()
did, _ := happydns.NewRandomIdentifier()
target := happydns.CheckTarget{UserId: uid.String(), DomainId: did.String()}
exec, err := engine.CreateExecution("test_discovery_publisher", target, nil)
if err != nil {
t.Fatalf("CreateExecution: %v", err)
}
if _, err := engine.RunExecution(context.Background(), exec, nil, nil); err != nil {
t.Fatalf("RunExecution: %v", err)
}
got, err := store.ListDiscoveryEntriesByProducer("test_discovery_publisher", target)
if err != nil {
t.Fatalf("ListDiscoveryEntriesByProducer: %v", err)
}
if len(got) != 2 {
t.Fatalf("expected 2 published entries, got %d", len(got))
}
// A second run with the provider publishing no entries should clear
// the previously-published set (replace-by-source).
provider.entries = nil
exec2, _ := engine.CreateExecution("test_discovery_publisher", target, nil)
if _, err := engine.RunExecution(context.Background(), exec2, nil, nil); err != nil {
t.Fatalf("RunExecution (empty): %v", err)
}
gotAfter, _ := store.ListDiscoveryEntriesByProducer("test_discovery_publisher", target)
if len(gotAfter) != 0 {
t.Fatalf("expected 0 entries after empty run, got %d", len(gotAfter))
}
}
// testCheckRuleReadingKey evaluates by calling obs.Get on a specific key,
// so the rule triggers collection of that observation provider.
type testCheckRuleReadingKey struct {
name string
key happydns.ObservationKey
}
func (r *testCheckRuleReadingKey) Name() string { return r.name }
func (r *testCheckRuleReadingKey) Description() string { return "test rule: " + r.name }
func (r *testCheckRuleReadingKey) Evaluate(ctx context.Context, obs happydns.ObservationGetter, opts happydns.CheckerOptions) []happydns.CheckState {
var data map[string]any
if err := obs.Get(ctx, r.key, &data); err != nil {
return []happydns.CheckState{{Status: happydns.StatusError, Message: err.Error()}}
}
return []happydns.CheckState{{Status: happydns.StatusOK}}
}
// consumingProvider reads AutoFillDiscoveryEntries from its options and
// stores the count so the test can verify entries were injected.
type consumingProvider struct {
key happydns.ObservationKey
lastCount int
lastRefs []string
}
func (p *consumingProvider) Key() happydns.ObservationKey { return p.key }
func (p *consumingProvider) Collect(ctx context.Context, opts happydns.CheckerOptions) (any, error) {
entries, _ := opts["consumed_entries"].([]happydns.DiscoveryEntry)
p.lastCount = len(entries)
p.lastRefs = nil
for _, e := range entries {
p.lastRefs = append(p.lastRefs, e.Ref)
}
return map[string]any{"seen": p.lastCount}, nil
}
// discoveryCaptureRule stores the RelatedObservations it sees on its last
// evaluation, so tests can assert on GetRelated behavior. `key` is the
// observation it reads to trigger collection; `relatedKey` is the key it
// asks GetRelated about (typically the downstream consumer's key).
type discoveryCaptureRule struct {
name string
key happydns.ObservationKey
relatedKey happydns.ObservationKey
lastRelated []happydns.RelatedObservation
}
func (r *discoveryCaptureRule) Name() string { return r.name }
func (r *discoveryCaptureRule) Description() string { return "capture related: " + r.name }
func (r *discoveryCaptureRule) Evaluate(ctx context.Context, obs happydns.ObservationGetter, opts happydns.CheckerOptions) []happydns.CheckState {
var data map[string]any
_ = obs.Get(ctx, r.key, &data)
related, _ := obs.GetRelated(ctx, r.relatedKey)
r.lastRelated = related
return []happydns.CheckState{{Status: happydns.StatusOK}}
}
func TestCheckerEngine_CrossCheckerDiscovery(t *testing.T) {
store, err := inmemory.Instantiate()
if err != nil {
t.Fatalf("Instantiate: %v", err)
}
// Producer: publishes one entry per run.
producer := &discoveringProvider{
key: "prod_obs",
entries: []happydns.DiscoveryEntry{
{Type: "t.v1", Ref: "host:443", Payload: json.RawMessage(`{"port":443}`)},
},
}
checker.RegisterObservationProvider(producer)
// The producer rule reads its own observation (prod_obs) but queries
// related observations by the consumer's key (cons_obs) — that is the
// key under which downstream checkers stored their findings about the
// producer's entries.
producerRule := &discoveryCaptureRule{name: "producer_rule", key: "prod_obs", relatedKey: "cons_obs"}
checker.RegisterChecker(&happydns.CheckerDefinition{
ID: "xchk_producer",
Name: "Cross-checker Producer",
Availability: happydns.CheckerAvailability{ApplyToDomain: true},
ObservationKeys: []happydns.ObservationKey{"prod_obs"},
Rules: []happydns.CheckRule{producerRule},
})
// Consumer: reads AutoFillDiscoveryEntries, produces an observation.
consumer := &consumingProvider{key: "cons_obs"}
checker.RegisterObservationProvider(consumer)
checker.RegisterChecker(&happydns.CheckerDefinition{
ID: "xchk_consumer",
Name: "Cross-checker Consumer",
Availability: happydns.CheckerAvailability{ApplyToDomain: true},
Options: happydns.CheckerOptionsDocumentation{
DomainOpts: []happydns.CheckerOptionDocumentation{
{Id: "consumed_entries", Type: "array", AutoFill: happydns.AutoFillDiscoveryEntries},
},
},
ObservationKeys: []happydns.ObservationKey{"cons_obs"},
Rules: []happydns.CheckRule{&testCheckRuleReadingKey{name: "cons_rule", key: "cons_obs"}},
})
optionsUC := checkerUC.NewCheckerOptionsUsecase(store, nil).WithDiscoveryEntryStore(store)
engine := checkerUC.NewCheckerEngine(optionsUC, store, store, store, store, store, store)
uid, _ := happydns.NewRandomIdentifier()
did, _ := happydns.NewRandomIdentifier()
target := happydns.CheckTarget{UserId: uid.String(), DomainId: did.String()}
// 1) Producer runs first, publishing entries.
prodExec, _ := engine.CreateExecution("xchk_producer", target, nil)
if _, err := engine.RunExecution(context.Background(), prodExec, nil, nil); err != nil {
t.Fatalf("producer first run: %v", err)
}
// 2) Consumer runs: must see the producer's entry in its option.
consExec, _ := engine.CreateExecution("xchk_consumer", target, nil)
if _, err := engine.RunExecution(context.Background(), consExec, nil, nil); err != nil {
t.Fatalf("consumer run: %v", err)
}
if consumer.lastCount != 1 || len(consumer.lastRefs) != 1 || consumer.lastRefs[0] != "host:443" {
t.Fatalf("consumer did not receive AutoFillDiscoveryEntries: count=%d refs=%v", consumer.lastCount, consumer.lastRefs)
}
// 3) Producer runs again: its rule's GetRelated should surface the
// consumer's observation, referencing the original Ref.
prodExec2, _ := engine.CreateExecution("xchk_producer", target, nil)
if _, err := engine.RunExecution(context.Background(), prodExec2, nil, nil); err != nil {
t.Fatalf("producer second run: %v", err)
}
if len(producerRule.lastRelated) != 1 {
t.Fatalf("expected 1 related observation, got %d", len(producerRule.lastRelated))
}
rel := producerRule.lastRelated[0]
if rel.CheckerID != "xchk_consumer" || rel.Ref != "host:443" || rel.Key != "cons_obs" {
t.Fatalf("related observation has wrong metadata: %+v", rel)
}
}

View file

@ -83,8 +83,9 @@ func (u *CheckerOptionsUsecase) getScopedOptions(
// CheckerOptionsUsecase handles the resolution and persistence of checker options.
type CheckerOptionsUsecase struct {
store CheckerOptionsStorage
autoFillStore CheckAutoFillStorage
store CheckerOptionsStorage
autoFillStore CheckAutoFillStorage
discoveryStore DiscoveryEntryStorage
}
// NewCheckerOptionsUsecase creates a new CheckerOptionsUsecase.
@ -92,6 +93,15 @@ func NewCheckerOptionsUsecase(store CheckerOptionsStorage, autoFillStore CheckAu
return &CheckerOptionsUsecase{store: store, autoFillStore: autoFillStore}
}
// WithDiscoveryEntryStore enables AutoFillDiscoveryEntries: options fields
// declaring that auto-fill will be populated with the entries stored for the
// target (see docs/checker-discovery.md). Passing nil (or not calling this)
// keeps AutoFillDiscoveryEntries fields unfilled.
func (u *CheckerOptionsUsecase) WithDiscoveryEntryStore(store DiscoveryEntryStorage) *CheckerOptionsUsecase {
u.discoveryStore = store
return u
}
// GetCheckerOptionsPositional returns the raw positional options from all scope levels,
// ordered from least to most specific (admin < user < domain < service).
func (u *CheckerOptionsUsecase) GetCheckerOptionsPositional(
@ -578,16 +588,21 @@ func (u *CheckerOptionsUsecase) resolveAutoFill(
// BuildMergedCheckerOptionsWithAutoFill merges stored options, runtime overrides,
// and auto-fill values. Auto-fill values are applied last and always win.
//
// The second return value is the set of DiscoveryEntry records injected into
// AutoFillDiscoveryEntries fields (if any) — exposed so the engine can
// persist the consumer→entry lineage after the run completes. It is nil
// when no such field was auto-filled.
func (u *CheckerOptionsUsecase) BuildMergedCheckerOptionsWithAutoFill(
checkerName string,
userId *happydns.Identifier,
domainId *happydns.Identifier,
serviceId *happydns.Identifier,
runOpts happydns.CheckerOptions,
) (happydns.CheckerOptions, error) {
) (happydns.CheckerOptions, []*happydns.StoredDiscoveryEntry, error) {
positionals, err := u.store.GetCheckerConfiguration(checkerName, userId, domainId, serviceId)
if err != nil {
return nil, err
return nil, nil, err
}
def := checkerPkg.FindChecker(checkerName)
@ -620,6 +635,8 @@ func (u *CheckerOptionsUsecase) BuildMergedCheckerOptionsWithAutoFill(
}
}
var injectedEntries []*happydns.StoredDiscoveryEntry
// Resolve auto-fill values (always win).
if def != nil && len(meta.autoFillIds) > 0 {
target := happydns.CheckTarget{
@ -629,14 +646,52 @@ func (u *CheckerOptionsUsecase) BuildMergedCheckerOptionsWithAutoFill(
}
ctx, err := u.buildAutoFillContext(target)
if err != nil {
return nil, err
return nil, nil, err
}
// AutoFillDiscoveryEntries is resolved from a separate storage surface
// (the discovery entry index), loaded lazily on first encounter.
var discoveryEntries []*happydns.StoredDiscoveryEntry
var discoveryLoaded bool
for fieldId, autoFillKey := range meta.autoFillIds {
if autoFillKey == happydns.AutoFillDiscoveryEntries {
if !discoveryLoaded {
discoveryLoaded = true
if u.discoveryStore != nil {
discoveryEntries, err = u.discoveryStore.ListDiscoveryEntriesByTarget(target)
if err != nil {
return nil, nil, fmt.Errorf("loading discovery entries: %w", err)
}
if len(discoveryEntries) > 0 {
injectedEntries = discoveryEntries
}
}
}
merged[fieldId] = sdkEntries(discoveryEntries)
continue
}
if val, ok := ctx[autoFillKey]; ok {
merged[fieldId] = val
}
}
}
return merged, nil
return merged, injectedEntries, nil
}
// sdkEntries converts host-side StoredDiscoveryEntry values to the opaque
// SDK-level DiscoveryEntry form that is passed to consumer checkers. The
// producer/target namespacing is not exposed to the consumer — it would be
// meaningless in that contract.
func sdkEntries(stored []*happydns.StoredDiscoveryEntry) []happydns.DiscoveryEntry {
out := make([]happydns.DiscoveryEntry, 0, len(stored))
for _, e := range stored {
out = append(out, happydns.DiscoveryEntry{
Type: e.Type,
Ref: e.Ref,
Payload: e.Payload,
})
}
return out
}

View file

@ -131,8 +131,8 @@ type validatingRule struct {
func (r *validatingRule) Name() string { return r.name }
func (r *validatingRule) Description() string { return "validating rule" }
func (r *validatingRule) Evaluate(_ context.Context, _ happydns.ObservationGetter, _ happydns.CheckerOptions) happydns.CheckState {
return happydns.CheckState{Status: happydns.StatusOK}
func (r *validatingRule) Evaluate(_ context.Context, _ happydns.ObservationGetter, _ happydns.CheckerOptions) []happydns.CheckState {
return []happydns.CheckState{{Status: happydns.StatusOK}}
}
func (r *validatingRule) ValidateOptions(_ happydns.CheckerOptions) error {
return r.validateErr
@ -146,8 +146,8 @@ type ruleWithOptions struct {
func (r *ruleWithOptions) Name() string { return r.name }
func (r *ruleWithOptions) Description() string { return "rule with options" }
func (r *ruleWithOptions) Evaluate(_ context.Context, _ happydns.ObservationGetter, _ happydns.CheckerOptions) happydns.CheckState {
return happydns.CheckState{Status: happydns.StatusOK}
func (r *ruleWithOptions) Evaluate(_ context.Context, _ happydns.ObservationGetter, _ happydns.CheckerOptions) []happydns.CheckState {
return []happydns.CheckState{{Status: happydns.StatusOK}}
}
func (r *ruleWithOptions) Options() happydns.CheckerOptionsDocumentation {
return r.opts
@ -1331,7 +1331,7 @@ func TestBuildMergedCheckerOptionsWithAutoFill_InjectsValues(t *testing.T) {
uc := checkerUC.NewCheckerOptionsUsecase(optStore, afStore)
_ = uc.SetCheckerOptions("af_inject", uid, nil, nil, happydns.CheckerOptions{"user_opt": "hello"})
merged, err := uc.BuildMergedCheckerOptionsWithAutoFill("af_inject", uid, did, nil, nil)
merged, _, err := uc.BuildMergedCheckerOptionsWithAutoFill("af_inject", uid, did, nil, nil)
if err != nil {
t.Fatal(err)
}
@ -1370,7 +1370,7 @@ func TestBuildMergedCheckerOptionsWithAutoFill_OverridesRunOpts(t *testing.T) {
uc := checkerUC.NewCheckerOptionsUsecase(optStore, afStore)
// Even if runOpts tries to set the auto-fill field, auto-fill wins.
merged, err := uc.BuildMergedCheckerOptionsWithAutoFill("af_override", uid, did, nil,
merged, _, err := uc.BuildMergedCheckerOptionsWithAutoFill("af_override", uid, did, nil,
happydns.CheckerOptions{"dn": "user-provided.com."})
if err != nil {
t.Fatal(err)
@ -1535,7 +1535,7 @@ func TestBuildMergedCheckerOptionsWithAutoFill_NoOverrideBlocksRunOpts(t *testin
})
// RunOpts tries to override locked.
merged, err := uc.BuildMergedCheckerOptionsWithAutoFill("no_override_runopt", uid, nil, nil, happydns.CheckerOptions{
merged, _, err := uc.BuildMergedCheckerOptionsWithAutoFill("no_override_runopt", uid, nil, nil, happydns.CheckerOptions{
"locked": false,
})
if err != nil {

View file

@ -130,3 +130,33 @@ type ObservationCacheStorage interface {
GetCachedObservation(target happydns.CheckTarget, key happydns.ObservationKey) (*happydns.ObservationCacheEntry, error)
PutCachedObservation(target happydns.CheckTarget, key happydns.ObservationKey, entry *happydns.ObservationCacheEntry) error
}
// DiscoveryEntryStorage persists DiscoveryEntry records published by a
// producer checker for a given target. Entries are replaced atomically on
// each collection cycle (see docs/checker-discovery.md).
type DiscoveryEntryStorage interface {
// ListDiscoveryEntriesByTarget returns entries published for a target
// across all producers — used to fill AutoFillDiscoveryEntries options.
ListDiscoveryEntriesByTarget(target happydns.CheckTarget) ([]*happydns.StoredDiscoveryEntry, error)
// ListDiscoveryEntriesByProducer returns entries a producer published
// for a target — used to resolve GetRelated.
ListDiscoveryEntriesByProducer(producerID string, target happydns.CheckTarget) ([]*happydns.StoredDiscoveryEntry, error)
// ReplaceDiscoveryEntries atomically replaces the set of entries
// stored for (producerID, target).
ReplaceDiscoveryEntries(producerID string, target happydns.CheckTarget, entries []happydns.DiscoveryEntry) error
DeleteDiscoveryEntriesByProducer(producerID string, target happydns.CheckTarget) error
ListAllDiscoveryEntries() (happydns.Iterator[happydns.StoredDiscoveryEntry], error)
RestoreDiscoveryEntry(entry *happydns.StoredDiscoveryEntry) error
ClearDiscoveryEntries() error
}
// DiscoveryObservationStorage persists the lineage linking consumer-produced
// observations to the DiscoveryEntry records they covered.
type DiscoveryObservationStorage interface {
PutDiscoveryObservationRef(ref *happydns.DiscoveryObservationRef) error
ListDiscoveryObservationRefs(producerID string, target happydns.CheckTarget, ref string) ([]*happydns.DiscoveryObservationRef, error)
DeleteDiscoveryObservationRefsForSnapshot(snapshotID happydns.Identifier) error
ListAllDiscoveryObservationRefs() (happydns.Iterator[happydns.DiscoveryObservationRef], error)
RestoreDiscoveryObservationRef(ref *happydns.DiscoveryObservationRef) error
ClearDiscoveryObservationRefs() error
}

View file

@ -22,17 +22,19 @@
package happydns
type Backup struct {
Version int
Domains []*Domain
DomainsLogs map[string][]*DomainLog
Errors []string
Providers []*ProviderMessage
Sessions []*Session
Users []*User
UsersAuth UserAuths
Zones []*ZoneMessage
CheckerConfigurations []*CheckerOptionsPositional
CheckPlans []*CheckPlan
CheckEvaluations []*CheckEvaluation
Executions []*Execution
Version int
Domains []*Domain
DomainsLogs map[string][]*DomainLog
Errors []string
Providers []*ProviderMessage
Sessions []*Session
Users []*User
UsersAuth UserAuths
Zones []*ZoneMessage
CheckerConfigurations []*CheckerOptionsPositional
CheckPlans []*CheckPlan
CheckEvaluations []*CheckEvaluation
Executions []*Execution
DiscoveryEntries []*StoredDiscoveryEntry
DiscoveryObservationRefs []*DiscoveryObservationRef
}

View file

@ -50,11 +50,12 @@ const (
)
const (
AutoFillDomainName = sdk.AutoFillDomainName
AutoFillSubdomain = sdk.AutoFillSubdomain
AutoFillZone = sdk.AutoFillZone
AutoFillServiceType = sdk.AutoFillServiceType
AutoFillService = sdk.AutoFillService
AutoFillDomainName = sdk.AutoFillDomainName
AutoFillSubdomain = sdk.AutoFillSubdomain
AutoFillZone = sdk.AutoFillZone
AutoFillServiceType = sdk.AutoFillServiceType
AutoFillService = sdk.AutoFillService
AutoFillDiscoveryEntries = sdk.AutoFillDiscoveryEntries
)
type (
@ -84,6 +85,10 @@ type (
ExternalEvaluateRequest = sdk.ExternalEvaluateRequest
ExternalEvaluateResponse = sdk.ExternalEvaluateResponse
ExternalReportRequest = sdk.ExternalReportRequest
DiscoveryEntry = sdk.DiscoveryEntry
DiscoveryPublisher = sdk.DiscoveryPublisher
RelatedObservation = sdk.RelatedObservation
ReportContext = sdk.ReportContext
)
const (
@ -205,6 +210,32 @@ type ObservationCacheEntry struct {
CollectedAt time.Time `json:"collectedAt"`
}
// StoredDiscoveryEntry is the host-side persistent form of a DiscoveryEntry:
// the opaque SDK-level (Type, Ref, Payload) triple augmented with the
// producing checker and target — the namespace under which the host dedupes
// and replaces entries across collection cycles.
type StoredDiscoveryEntry struct {
ProducerID string `json:"producerId"`
Target CheckTarget `json:"target"`
Type string `json:"type"`
Ref string `json:"ref"`
Payload json.RawMessage `json:"payload,omitempty" swaggertype:"object"`
}
// DiscoveryObservationRef links a consumer's observation to a specific
// DiscoveryEntry it covered. It lets the host resolve
// ObservationGetter.GetRelated and ReportContext.Related without re-parsing
// snapshots.
type DiscoveryObservationRef struct {
ProducerID string `json:"producerId"`
Target CheckTarget `json:"target"`
Ref string `json:"ref"`
ConsumerID string `json:"consumerId"`
Key ObservationKey `json:"key"`
SnapshotID Identifier `json:"snapshotId" swaggertype:"string"`
CollectedAt time.Time `json:"collectedAt" format:"date-time"`
}
// ExecutionStatus represents the lifecycle state of an execution.
type ExecutionStatus int

View file

@ -45,6 +45,8 @@ var entityMap = map[string]string{
"CheckerOptionsStorage": "check_config",
"CheckEvaluationStorage": "check_evaluation",
"ExecutionStorage": "execution",
"DiscoveryEntryStorage": "discovery_entry",
"DiscoveryObservationStorage": "discovery_observation",
"ObservationCacheStorage": "observation_cache",
"ObservationSnapshotStorage": "observation_snapshot",
"SchedulerStateStorage": "scheduler_state",
@ -59,10 +61,11 @@ var entityMap = map[string]string{
// operationOverrides maps method names that don't follow the prefix convention.
var operationOverrides = map[string]string{
"AuthUserExists": "get",
"InsightsRun": "run",
"LastInsightsRun": "get",
"CreateOrUpdateUser": "update",
"AuthUserExists": "get",
"InsightsRun": "run",
"LastInsightsRun": "get",
"CreateOrUpdateUser": "update",
"ReplaceDiscoveryEntries": "update",
}
// skipMethods lists methods that should be passed through without instrumentation.

View file

@ -52,9 +52,14 @@
<tbody>
{#each evaluation.states as state}
<tr>
<td><code>{state.code ?? ""}</code></td>
<td>
<code>{state.rule ?? ""}</code>
{#if state.code}<small class="text-muted"> · {state.code}</small>{/if}
</td>
<td><Badge color={getStatusColor(state.status)}>{$t(getStatusI18nKey(state.status))}</Badge></td>
<td>{state.message ?? ""}</td>
<td>
{#if state.subject}<strong>{state.subject}</strong>{#if state.message}: {/if}{/if}{state.message ?? ""}
</td>
</tr>
{/each}
</tbody>