robots.txt and security.txt are independent requests to the same host;
fetch them in parallel so the collector's latency is the slower of the
two rather than their sum (each pays its own TLS handshake under the
keep-alive-disabled transport). Each goroutine writes its own variable
and the map is assembled after both finish, so no locking is needed.
The http.security_txt rule reported OK for any 200 response with a
non-empty body, so a soft-404 (status 200 + HTML body) served for
/.well-known/security.txt was misread as "published".
Capture the response Content-Type and count the RFC 9116 required
fields (Contact, Expires) in the body. OK now requires text/plain with
at least one Contact and exactly one Expires; a non-conforming 200
yields a new Warn http.security_txt.invalid explaining the defect.
Redirects are still followed and the final response is validated, per
RFC 9116 §3.
Probes 20 known-bad paths (/.env, /.git/config, /actuator/env, etc.)
that CT-log scanners hit immediately after a new certificate is issued.
Critical credential/source-leak paths raise StatusCrit; other exposed
paths raise StatusWarn; 401/403 responses raise StatusInfo.
Fixes: #1
