Add rspamd as a second spam filter alongside SpamAssassin
All checks were successful
continuous-integration/drone/push Build is passing
All checks were successful
continuous-integration/drone/push Build is passing
Closes: #36
This commit is contained in:
parent
8fda7746a1
commit
51321ecb1a
19 changed files with 513 additions and 28 deletions
|
|
@ -121,6 +121,7 @@ RUN echo "@edge https://dl-cdn.alpinelinux.org/alpine/edge/community" >> /etc/ap
|
||||||
perl-xml-libxml \
|
perl-xml-libxml \
|
||||||
postfix \
|
postfix \
|
||||||
postfix-pcre \
|
postfix-pcre \
|
||||||
|
rspamd \
|
||||||
spamassassin \
|
spamassassin \
|
||||||
spamassassin-client \
|
spamassassin-client \
|
||||||
supervisor \
|
supervisor \
|
||||||
|
|
@ -143,8 +144,11 @@ RUN mkdir -p /etc/happydeliver \
|
||||||
/var/lib/authentication_milter \
|
/var/lib/authentication_milter \
|
||||||
/var/spool/postfix/authentication_milter \
|
/var/spool/postfix/authentication_milter \
|
||||||
/var/spool/postfix/spamassassin \
|
/var/spool/postfix/spamassassin \
|
||||||
|
/var/spool/postfix/rspamd \
|
||||||
&& chown -R happydeliver:happydeliver /var/lib/happydeliver /var/log/happydeliver \
|
&& chown -R happydeliver:happydeliver /var/lib/happydeliver /var/log/happydeliver \
|
||||||
&& chown -R mail:mail /var/spool/postfix/authentication_milter /var/spool/postfix/spamassassin
|
&& chown -R mail:mail /var/spool/postfix/authentication_milter /var/spool/postfix/spamassassin \
|
||||||
|
&& chown rspamd:mail /var/spool/postfix/rspamd \
|
||||||
|
&& chmod 750 /var/spool/postfix/rspamd
|
||||||
|
|
||||||
# Copy the built application
|
# Copy the built application
|
||||||
COPY --from=builder /build/happyDeliver /usr/local/bin/happyDeliver
|
COPY --from=builder /build/happyDeliver /usr/local/bin/happyDeliver
|
||||||
|
|
@ -154,6 +158,7 @@ RUN chmod +x /usr/local/bin/happyDeliver
|
||||||
COPY docker/postfix/ /etc/postfix/
|
COPY docker/postfix/ /etc/postfix/
|
||||||
COPY docker/authentication_milter/authentication_milter.json /etc/authentication_milter.json
|
COPY docker/authentication_milter/authentication_milter.json /etc/authentication_milter.json
|
||||||
COPY docker/spamassassin/ /etc/mail/spamassassin/
|
COPY docker/spamassassin/ /etc/mail/spamassassin/
|
||||||
|
COPY docker/rspamd/local.d/ /etc/rspamd/local.d/
|
||||||
COPY docker/supervisor/ /etc/supervisor/
|
COPY docker/supervisor/ /etc/supervisor/
|
||||||
COPY docker/entrypoint.sh /entrypoint.sh
|
COPY docker/entrypoint.sh /entrypoint.sh
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -6,7 +6,7 @@ An open-source email deliverability testing platform that analyzes test emails a
|
||||||
|
|
||||||
## Features
|
## Features
|
||||||
|
|
||||||
- **Complete Email Analysis**: Analyzes SPF, DKIM, DMARC, BIMI, ARC, SpamAssassin scores, DNS records, blacklist status, content quality, and more
|
- **Complete Email Analysis**: Analyzes SPF, DKIM, DMARC, BIMI, ARC, SpamAssassin and rspamd scores, DNS records, blacklist status, content quality, and more
|
||||||
- **REST API**: Full-featured API for creating tests and retrieving reports
|
- **REST API**: Full-featured API for creating tests and retrieving reports
|
||||||
- **LMTP Server**: Built-in LMTP server for seamless MTA integration
|
- **LMTP Server**: Built-in LMTP server for seamless MTA integration
|
||||||
- **Scoring System**: Gives A to F grades and scoring with weighted factors across dns, authentication, spam, blacklists, content, and headers
|
- **Scoring System**: Gives A to F grades and scoring with weighted factors across dns, authentication, spam, blacklists, content, and headers
|
||||||
|
|
@ -26,6 +26,7 @@ The easiest way to run happyDeliver is using the all-in-one Docker container tha
|
||||||
- **Postfix MTA**: Receives emails on port 25
|
- **Postfix MTA**: Receives emails on port 25
|
||||||
- **authentication_milter**: Entreprise grade email authentication
|
- **authentication_milter**: Entreprise grade email authentication
|
||||||
- **SpamAssassin**: Spam scoring and analysis
|
- **SpamAssassin**: Spam scoring and analysis
|
||||||
|
- **rspamd**: Second spam filter for cross-validated scoring
|
||||||
- **happyDeliver API**: REST API server on port 8080
|
- **happyDeliver API**: REST API server on port 8080
|
||||||
- **SQLite Database**: Persistent storage for tests and reports
|
- **SQLite Database**: Persistent storage for tests and reports
|
||||||
|
|
||||||
|
|
@ -162,7 +163,7 @@ The server will start on `http://localhost:8080` by default.
|
||||||
|
|
||||||
#### 3. Integrate with your existing e-mail setup
|
#### 3. Integrate with your existing e-mail setup
|
||||||
|
|
||||||
It is expected your setup annotate the email with eg. opendkim, spamassassin, ...
|
It is expected your setup annotate the email with eg. opendkim, spamassassin, rspamd, ...
|
||||||
happyDeliver will not perform thoses checks, it relies instead on standard software to have real world annotations.
|
happyDeliver will not perform thoses checks, it relies instead on standard software to have real world annotations.
|
||||||
|
|
||||||
Choose one of the following way to integrate happyDeliver in your existing setup:
|
Choose one of the following way to integrate happyDeliver in your existing setup:
|
||||||
|
|
@ -269,7 +270,7 @@ The deliverability score is calculated from A to F based on:
|
||||||
- **Authentication**: IPRev, SPF, DKIM, DMARC, BIMI and ARC validation
|
- **Authentication**: IPRev, SPF, DKIM, DMARC, BIMI and ARC validation
|
||||||
- **Blacklist**: RBL/DNSBL checks
|
- **Blacklist**: RBL/DNSBL checks
|
||||||
- **Headers**: Required headers, MIME structure, Domain alignment
|
- **Headers**: Required headers, MIME structure, Domain alignment
|
||||||
- **Spam**: SpamAssassin score
|
- **Spam**: SpamAssassin and rspamd scores (combined 50/50)
|
||||||
- **Content**: HTML quality, links, images, unsubscribe
|
- **Content**: HTML quality, links, images, unsubscribe
|
||||||
|
|
||||||
## Funding
|
## Funding
|
||||||
|
|
|
||||||
|
|
@ -333,6 +333,8 @@ components:
|
||||||
$ref: '#/components/schemas/AuthenticationResults'
|
$ref: '#/components/schemas/AuthenticationResults'
|
||||||
spamassassin:
|
spamassassin:
|
||||||
$ref: '#/components/schemas/SpamAssassinResult'
|
$ref: '#/components/schemas/SpamAssassinResult'
|
||||||
|
rspamd:
|
||||||
|
$ref: '#/components/schemas/RspamdResult'
|
||||||
dns_results:
|
dns_results:
|
||||||
$ref: '#/components/schemas/DNSResults'
|
$ref: '#/components/schemas/DNSResults'
|
||||||
blacklists:
|
blacklists:
|
||||||
|
|
@ -401,7 +403,7 @@ components:
|
||||||
type: integer
|
type: integer
|
||||||
minimum: 0
|
minimum: 0
|
||||||
maximum: 100
|
maximum: 100
|
||||||
description: SpamAssassin score (in percentage)
|
description: Spam filter score (SpamAssassin + rspamd combined, in percentage)
|
||||||
example: 15
|
example: 15
|
||||||
spam_grade:
|
spam_grade:
|
||||||
type: string
|
type: string
|
||||||
|
|
@ -843,6 +845,17 @@ components:
|
||||||
- is_spam
|
- is_spam
|
||||||
- test_details
|
- test_details
|
||||||
properties:
|
properties:
|
||||||
|
deliverability_score:
|
||||||
|
type: integer
|
||||||
|
minimum: 0
|
||||||
|
maximum: 100
|
||||||
|
description: SpamAssassin deliverability score (0-100, higher is better)
|
||||||
|
example: 80
|
||||||
|
deliverability_grade:
|
||||||
|
type: string
|
||||||
|
enum: [A+, A, B, C, D, E, F]
|
||||||
|
description: Letter grade for SpamAssassin deliverability score
|
||||||
|
example: "B"
|
||||||
version:
|
version:
|
||||||
type: string
|
type: string
|
||||||
description: SpamAssassin version
|
description: SpamAssassin version
|
||||||
|
|
@ -905,6 +918,78 @@ components:
|
||||||
description: Human-readable description of what this test checks
|
description: Human-readable description of what this test checks
|
||||||
example: "Bayes spam probability is 0 to 1%"
|
example: "Bayes spam probability is 0 to 1%"
|
||||||
|
|
||||||
|
RspamdResult:
|
||||||
|
type: object
|
||||||
|
required:
|
||||||
|
- score
|
||||||
|
- threshold
|
||||||
|
- is_spam
|
||||||
|
- symbols
|
||||||
|
properties:
|
||||||
|
deliverability_score:
|
||||||
|
type: integer
|
||||||
|
minimum: 0
|
||||||
|
maximum: 100
|
||||||
|
description: rspamd deliverability score (0-100, higher is better)
|
||||||
|
example: 85
|
||||||
|
deliverability_grade:
|
||||||
|
type: string
|
||||||
|
enum: [A+, A, B, C, D, E, F]
|
||||||
|
description: Letter grade for rspamd deliverability score
|
||||||
|
example: "A"
|
||||||
|
score:
|
||||||
|
type: number
|
||||||
|
format: float
|
||||||
|
description: rspamd spam score
|
||||||
|
example: -3.91
|
||||||
|
threshold:
|
||||||
|
type: number
|
||||||
|
format: float
|
||||||
|
description: Score threshold for spam classification
|
||||||
|
example: 15.0
|
||||||
|
action:
|
||||||
|
type: string
|
||||||
|
description: rspamd action (no action, add header, rewrite subject, soft reject, reject)
|
||||||
|
example: "no action"
|
||||||
|
is_spam:
|
||||||
|
type: boolean
|
||||||
|
description: Whether message is classified as spam (action is reject or soft reject)
|
||||||
|
example: false
|
||||||
|
server:
|
||||||
|
type: string
|
||||||
|
description: rspamd server that processed the message
|
||||||
|
example: "rspamd.example.com"
|
||||||
|
symbols:
|
||||||
|
type: object
|
||||||
|
additionalProperties:
|
||||||
|
$ref: '#/components/schemas/RspamdSymbol'
|
||||||
|
description: Map of triggered rspamd symbols to their details
|
||||||
|
example:
|
||||||
|
BAYES_HAM:
|
||||||
|
name: "BAYES_HAM"
|
||||||
|
score: -1.9
|
||||||
|
params: "0.02"
|
||||||
|
|
||||||
|
RspamdSymbol:
|
||||||
|
type: object
|
||||||
|
required:
|
||||||
|
- name
|
||||||
|
- score
|
||||||
|
properties:
|
||||||
|
name:
|
||||||
|
type: string
|
||||||
|
description: Symbol name
|
||||||
|
example: "BAYES_HAM"
|
||||||
|
score:
|
||||||
|
type: number
|
||||||
|
format: float
|
||||||
|
description: Score contribution of this symbol
|
||||||
|
example: -1.9
|
||||||
|
params:
|
||||||
|
type: string
|
||||||
|
description: Symbol parameters or options
|
||||||
|
example: "0.02"
|
||||||
|
|
||||||
DNSResults:
|
DNSResults:
|
||||||
type: object
|
type: object
|
||||||
required:
|
required:
|
||||||
|
|
|
||||||
|
|
@ -15,6 +15,10 @@ mkdir -p /var/spool/postfix/authentication_milter
|
||||||
chown mail:mail /var/spool/postfix/authentication_milter
|
chown mail:mail /var/spool/postfix/authentication_milter
|
||||||
chmod 750 /var/spool/postfix/authentication_milter
|
chmod 750 /var/spool/postfix/authentication_milter
|
||||||
|
|
||||||
|
mkdir -p /var/spool/postfix/rspamd
|
||||||
|
chown rspamd:mail /var/spool/postfix/rspamd
|
||||||
|
chmod 750 /var/spool/postfix/rspamd
|
||||||
|
|
||||||
# Create log directory
|
# Create log directory
|
||||||
mkdir -p /var/log/happydeliver /var/cache/authentication_milter /var/spool/authentication_milter /var/lib/authentication_milter /run/authentication_milter
|
mkdir -p /var/log/happydeliver /var/cache/authentication_milter /var/spool/authentication_milter /var/lib/authentication_milter /run/authentication_milter
|
||||||
chown happydeliver:happydeliver /var/log/happydeliver
|
chown happydeliver:happydeliver /var/log/happydeliver
|
||||||
|
|
|
||||||
|
|
@ -28,7 +28,7 @@ transport_maps = pcre:/etc/postfix/transport_maps
|
||||||
# OpenDKIM for DKIM verification
|
# OpenDKIM for DKIM verification
|
||||||
milter_default_action = accept
|
milter_default_action = accept
|
||||||
milter_protocol = 6
|
milter_protocol = 6
|
||||||
smtpd_milters = unix:/var/spool/postfix/authentication_milter/authentication_milter.sock unix:/var/spool/postfix/spamassassin/spamass-milter.sock
|
smtpd_milters = unix:/var/spool/postfix/authentication_milter/authentication_milter.sock unix:/var/spool/postfix/spamassassin/spamass-milter.sock unix:/var/spool/postfix/rspamd/rspamd-milter.sock
|
||||||
non_smtpd_milters = $smtpd_milters
|
non_smtpd_milters = $smtpd_milters
|
||||||
|
|
||||||
# SPF policy checking
|
# SPF policy checking
|
||||||
|
|
|
||||||
5
docker/rspamd/local.d/actions.conf
Normal file
5
docker/rspamd/local.d/actions.conf
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
no_action = 0;
|
||||||
|
reject = null;
|
||||||
|
add_header = null;
|
||||||
|
rewrite_subject = null;
|
||||||
|
greylist = null;
|
||||||
5
docker/rspamd/local.d/milter_headers.conf
Normal file
5
docker/rspamd/local.d/milter_headers.conf
Normal file
|
|
@ -0,0 +1,5 @@
|
||||||
|
# Add "extended Rspamd headers"
|
||||||
|
extended_spam_headers = true;
|
||||||
|
|
||||||
|
skip_local = false;
|
||||||
|
skip_authenticated = false;
|
||||||
3
docker/rspamd/local.d/options.inc
Normal file
3
docker/rspamd/local.d/options.inc
Normal file
|
|
@ -0,0 +1,3 @@
|
||||||
|
# rspamd options for happyDeliver
|
||||||
|
# Disable Bayes learning to keep the setup stateless
|
||||||
|
use_redis = false;
|
||||||
6
docker/rspamd/local.d/worker-proxy.inc
Normal file
6
docker/rspamd/local.d/worker-proxy.inc
Normal file
|
|
@ -0,0 +1,6 @@
|
||||||
|
# Enable rspamd milter proxy worker via Unix socket for Postfix integration
|
||||||
|
bind_socket = "/var/spool/postfix/rspamd/rspamd-milter.sock mode=0660 owner=rspamd group=mail";
|
||||||
|
upstream "local" {
|
||||||
|
default = yes;
|
||||||
|
self_scan = yes;
|
||||||
|
}
|
||||||
|
|
@ -33,6 +33,16 @@ stderr_logfile=/var/log/happydeliver/authentication_milter.log
|
||||||
user=mail
|
user=mail
|
||||||
group=mail
|
group=mail
|
||||||
|
|
||||||
|
# rspamd spam filter
|
||||||
|
[program:rspamd]
|
||||||
|
command=/usr/bin/rspamd -f -u rspamd -g mail
|
||||||
|
autostart=true
|
||||||
|
autorestart=true
|
||||||
|
priority=11
|
||||||
|
stdout_logfile=/var/log/happydeliver/rspamd.log
|
||||||
|
stderr_logfile=/var/log/happydeliver/rspamd_error.log
|
||||||
|
user=root
|
||||||
|
|
||||||
# SpamAssassin daemon
|
# SpamAssassin daemon
|
||||||
[program:spamd]
|
[program:spamd]
|
||||||
command=/usr/sbin/spamd --max-children 5 --helper-home-dir /var/lib/spamassassin --syslog stderr --pidfile /var/run/spamd.pid
|
command=/usr/sbin/spamd --max-children 5 --helper-home-dir /var/lib/spamassassin --syslog stderr --pidfile /var/run/spamd.pid
|
||||||
|
|
|
||||||
|
|
@ -264,6 +264,26 @@ func (e *EmailMessage) GetSpamAssassinHeaders() map[string]string {
|
||||||
return headers
|
return headers
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// GetRspamdHeaders extracts rspamd-related headers
|
||||||
|
func (e *EmailMessage) GetRspamdHeaders() map[string]string {
|
||||||
|
headers := make(map[string]string)
|
||||||
|
|
||||||
|
rspamdHeaders := []string{
|
||||||
|
"X-Spamd-Result",
|
||||||
|
"X-Rspamd-Score",
|
||||||
|
"X-Rspamd-Action",
|
||||||
|
"X-Rspamd-Server",
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, headerName := range rspamdHeaders {
|
||||||
|
if value := e.Header.Get(headerName); value != "" {
|
||||||
|
headers[headerName] = value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return headers
|
||||||
|
}
|
||||||
|
|
||||||
// GetTextParts returns all text/plain parts
|
// GetTextParts returns all text/plain parts
|
||||||
func (e *EmailMessage) GetTextParts() []MessagePart {
|
func (e *EmailMessage) GetTextParts() []MessagePart {
|
||||||
return filterParts(e.Parts, func(p MessagePart) bool {
|
return filterParts(e.Parts, func(p MessagePart) bool {
|
||||||
|
|
|
||||||
|
|
@ -33,6 +33,7 @@ import (
|
||||||
type ReportGenerator struct {
|
type ReportGenerator struct {
|
||||||
authAnalyzer *AuthenticationAnalyzer
|
authAnalyzer *AuthenticationAnalyzer
|
||||||
spamAnalyzer *SpamAssassinAnalyzer
|
spamAnalyzer *SpamAssassinAnalyzer
|
||||||
|
rspamdAnalyzer *RspamdAnalyzer
|
||||||
dnsAnalyzer *DNSAnalyzer
|
dnsAnalyzer *DNSAnalyzer
|
||||||
rblChecker *RBLChecker
|
rblChecker *RBLChecker
|
||||||
contentAnalyzer *ContentAnalyzer
|
contentAnalyzer *ContentAnalyzer
|
||||||
|
|
@ -49,6 +50,7 @@ func NewReportGenerator(
|
||||||
return &ReportGenerator{
|
return &ReportGenerator{
|
||||||
authAnalyzer: NewAuthenticationAnalyzer(),
|
authAnalyzer: NewAuthenticationAnalyzer(),
|
||||||
spamAnalyzer: NewSpamAssassinAnalyzer(),
|
spamAnalyzer: NewSpamAssassinAnalyzer(),
|
||||||
|
rspamdAnalyzer: NewRspamdAnalyzer(),
|
||||||
dnsAnalyzer: NewDNSAnalyzer(dnsTimeout),
|
dnsAnalyzer: NewDNSAnalyzer(dnsTimeout),
|
||||||
rblChecker: NewRBLChecker(dnsTimeout, rbls, checkAllIPs),
|
rblChecker: NewRBLChecker(dnsTimeout, rbls, checkAllIPs),
|
||||||
contentAnalyzer: NewContentAnalyzer(httpTimeout),
|
contentAnalyzer: NewContentAnalyzer(httpTimeout),
|
||||||
|
|
@ -65,6 +67,7 @@ type AnalysisResults struct {
|
||||||
Headers *api.HeaderAnalysis
|
Headers *api.HeaderAnalysis
|
||||||
RBL *RBLResults
|
RBL *RBLResults
|
||||||
SpamAssassin *api.SpamAssassinResult
|
SpamAssassin *api.SpamAssassinResult
|
||||||
|
Rspamd *api.RspamdResult
|
||||||
}
|
}
|
||||||
|
|
||||||
// AnalyzeEmail performs complete email analysis
|
// AnalyzeEmail performs complete email analysis
|
||||||
|
|
@ -79,6 +82,7 @@ func (r *ReportGenerator) AnalyzeEmail(email *EmailMessage) *AnalysisResults {
|
||||||
results.DNS = r.dnsAnalyzer.AnalyzeDNS(email, results.Authentication, results.Headers)
|
results.DNS = r.dnsAnalyzer.AnalyzeDNS(email, results.Authentication, results.Headers)
|
||||||
results.RBL = r.rblChecker.CheckEmail(email)
|
results.RBL = r.rblChecker.CheckEmail(email)
|
||||||
results.SpamAssassin = r.spamAnalyzer.AnalyzeSpamAssassin(email)
|
results.SpamAssassin = r.spamAnalyzer.AnalyzeSpamAssassin(email)
|
||||||
|
results.Rspamd = r.rspamdAnalyzer.AnalyzeRspamd(email)
|
||||||
results.Content = r.contentAnalyzer.AnalyzeContent(email)
|
results.Content = r.contentAnalyzer.AnalyzeContent(email)
|
||||||
|
|
||||||
return results
|
return results
|
||||||
|
|
@ -134,10 +138,26 @@ func (r *ReportGenerator) GenerateReport(testID uuid.UUID, results *AnalysisResu
|
||||||
blacklistScore, blacklistGrade = r.rblChecker.CalculateRBLScore(results.RBL)
|
blacklistScore, blacklistGrade = r.rblChecker.CalculateRBLScore(results.RBL)
|
||||||
}
|
}
|
||||||
|
|
||||||
spamScore := 0
|
saScore, saGrade := r.spamAnalyzer.CalculateSpamAssassinScore(results.SpamAssassin)
|
||||||
|
rspamdScore, rspamdGrade := r.rspamdAnalyzer.CalculateRspamdScore(results.Rspamd)
|
||||||
|
|
||||||
|
// Combine SpamAssassin and rspamd scores 50/50.
|
||||||
|
// If only one filter ran (the other returns "" grade), use that filter's score alone.
|
||||||
|
var spamScore int
|
||||||
var spamGrade string
|
var spamGrade string
|
||||||
if results.SpamAssassin != nil {
|
switch {
|
||||||
spamScore, spamGrade = r.spamAnalyzer.CalculateSpamAssassinScore(results.SpamAssassin)
|
case saGrade == "" && rspamdGrade == "":
|
||||||
|
spamScore = 0
|
||||||
|
spamGrade = ""
|
||||||
|
case saGrade == "":
|
||||||
|
spamScore = rspamdScore
|
||||||
|
spamGrade = rspamdGrade
|
||||||
|
case rspamdGrade == "":
|
||||||
|
spamScore = saScore
|
||||||
|
spamGrade = saGrade
|
||||||
|
default:
|
||||||
|
spamScore = (saScore + rspamdScore) / 2
|
||||||
|
spamGrade = MinGrade(saGrade, rspamdGrade)
|
||||||
}
|
}
|
||||||
|
|
||||||
report.Summary = &api.ScoreSummary{
|
report.Summary = &api.ScoreSummary{
|
||||||
|
|
@ -177,9 +197,22 @@ func (r *ReportGenerator) GenerateReport(testID uuid.UUID, results *AnalysisResu
|
||||||
report.Blacklists = &results.RBL.Checks
|
report.Blacklists = &results.RBL.Checks
|
||||||
}
|
}
|
||||||
|
|
||||||
// Add SpamAssassin result
|
// Add SpamAssassin result with individual deliverability score
|
||||||
|
if results.SpamAssassin != nil {
|
||||||
|
saGradeTyped := api.SpamAssassinResultDeliverabilityGrade(saGrade)
|
||||||
|
results.SpamAssassin.DeliverabilityScore = api.PtrTo(saScore)
|
||||||
|
results.SpamAssassin.DeliverabilityGrade = &saGradeTyped
|
||||||
|
}
|
||||||
report.Spamassassin = results.SpamAssassin
|
report.Spamassassin = results.SpamAssassin
|
||||||
|
|
||||||
|
// Add rspamd result with individual deliverability score
|
||||||
|
if results.Rspamd != nil {
|
||||||
|
rspamdGradeTyped := api.RspamdResultDeliverabilityGrade(rspamdGrade)
|
||||||
|
results.Rspamd.DeliverabilityScore = api.PtrTo(rspamdScore)
|
||||||
|
results.Rspamd.DeliverabilityGrade = &rspamdGradeTyped
|
||||||
|
}
|
||||||
|
report.Rspamd = results.Rspamd
|
||||||
|
|
||||||
// Add raw headers
|
// Add raw headers
|
||||||
if results.Email != nil && results.Email.RawHeaders != "" {
|
if results.Email != nil && results.Email.RawHeaders != "" {
|
||||||
report.RawHeaders = &results.Email.RawHeaders
|
report.RawHeaders = &results.Email.RawHeaders
|
||||||
|
|
|
||||||
152
pkg/analyzer/rspamd.go
Normal file
152
pkg/analyzer/rspamd.go
Normal file
|
|
@ -0,0 +1,152 @@
|
||||||
|
// This file is part of the happyDeliver (R) project.
|
||||||
|
// Copyright (c) 2026 happyDomain
|
||||||
|
// Authors: Pierre-Olivier Mercier, et al.
|
||||||
|
//
|
||||||
|
// This program is offered under a commercial and under the AGPL license.
|
||||||
|
// For commercial licensing, contact us at <contact@happydomain.org>.
|
||||||
|
//
|
||||||
|
// For AGPL licensing:
|
||||||
|
// This program is free software: you can redistribute it and/or modify
|
||||||
|
// it under the terms of the GNU Affero General Public License as published by
|
||||||
|
// the Free Software Foundation, either version 3 of the License, or
|
||||||
|
// (at your option) any later version.
|
||||||
|
//
|
||||||
|
// This program is distributed in the hope that it will be useful,
|
||||||
|
// but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||||||
|
// MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||||||
|
// GNU Affero General Public License for more details.
|
||||||
|
//
|
||||||
|
// You should have received a copy of the GNU Affero General Public License
|
||||||
|
// along with this program. If not, see <https://www.gnu.org/licenses/>.
|
||||||
|
|
||||||
|
package analyzer
|
||||||
|
|
||||||
|
import (
|
||||||
|
"math"
|
||||||
|
"regexp"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"git.happydns.org/happyDeliver/internal/api"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Default rspamd action thresholds (rspamd built-in defaults)
|
||||||
|
const (
|
||||||
|
rspamdDefaultRejectThreshold float32 = 15
|
||||||
|
rspamdDefaultAddHeaderThreshold float32 = 6
|
||||||
|
)
|
||||||
|
|
||||||
|
// RspamdAnalyzer analyzes rspamd results from email headers
|
||||||
|
type RspamdAnalyzer struct{}
|
||||||
|
|
||||||
|
// NewRspamdAnalyzer creates a new rspamd analyzer
|
||||||
|
func NewRspamdAnalyzer() *RspamdAnalyzer {
|
||||||
|
return &RspamdAnalyzer{}
|
||||||
|
}
|
||||||
|
|
||||||
|
// AnalyzeRspamd extracts and analyzes rspamd results from email headers
|
||||||
|
func (a *RspamdAnalyzer) AnalyzeRspamd(email *EmailMessage) *api.RspamdResult {
|
||||||
|
headers := email.GetRspamdHeaders()
|
||||||
|
if len(headers) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
result := &api.RspamdResult{
|
||||||
|
Symbols: make(map[string]api.RspamdSymbol),
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse X-Spamd-Result header (primary source for score, threshold, and symbols)
|
||||||
|
// Format: "default: False [-3.91 / 15.00];\n\tSYMBOL(score)[params]; ..."
|
||||||
|
if spamdResult, ok := headers["X-Spamd-Result"]; ok {
|
||||||
|
a.parseSpamdResult(spamdResult, result)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse X-Rspamd-Score as override/fallback for score
|
||||||
|
if scoreHeader, ok := headers["X-Rspamd-Score"]; ok {
|
||||||
|
if score, err := strconv.ParseFloat(strings.TrimSpace(scoreHeader), 64); err == nil {
|
||||||
|
result.Score = float32(score)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse X-Rspamd-Server
|
||||||
|
if serverHeader, ok := headers["X-Rspamd-Server"]; ok {
|
||||||
|
server := strings.TrimSpace(serverHeader)
|
||||||
|
result.Server = &server
|
||||||
|
}
|
||||||
|
|
||||||
|
// Derive IsSpam from score vs reject threshold.
|
||||||
|
if result.Threshold > 0 {
|
||||||
|
result.IsSpam = result.Score >= result.Threshold
|
||||||
|
} else {
|
||||||
|
result.IsSpam = result.Score >= rspamdDefaultAddHeaderThreshold
|
||||||
|
}
|
||||||
|
|
||||||
|
return result
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseSpamdResult parses the X-Spamd-Result header
|
||||||
|
// Format: "default: False [-3.91 / 15.00];\n\tSYMBOL(score)[params]; ..."
|
||||||
|
func (a *RspamdAnalyzer) parseSpamdResult(header string, result *api.RspamdResult) {
|
||||||
|
// Extract score and threshold from the first line
|
||||||
|
// e.g. "default: False [-3.91 / 15.00]"
|
||||||
|
scoreRe := regexp.MustCompile(`\[\s*(-?\d+\.?\d*)\s*/\s*(-?\d+\.?\d*)\s*\]`)
|
||||||
|
if matches := scoreRe.FindStringSubmatch(header); len(matches) > 2 {
|
||||||
|
if score, err := strconv.ParseFloat(matches[1], 64); err == nil {
|
||||||
|
result.Score = float32(score)
|
||||||
|
}
|
||||||
|
if threshold, err := strconv.ParseFloat(matches[2], 64); err == nil {
|
||||||
|
result.Threshold = float32(threshold)
|
||||||
|
|
||||||
|
// No threshold? use default AddHeaderThreshold
|
||||||
|
if result.Threshold <= 0 {
|
||||||
|
result.Threshold = rspamdDefaultAddHeaderThreshold
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse is_spam from header (before we may get action from X-Rspamd-Action)
|
||||||
|
firstLine := strings.SplitN(header, ";", 2)[0]
|
||||||
|
if strings.Contains(firstLine, ": True") || strings.Contains(firstLine, ": true") {
|
||||||
|
result.IsSpam = true
|
||||||
|
}
|
||||||
|
|
||||||
|
// Parse symbols: SYMBOL(score)[params]
|
||||||
|
// Each symbol entry is separated by ";"
|
||||||
|
symbolRe := regexp.MustCompile(`(\w+)\((-?\d+\.?\d*)\)(?:\[([^\]]*)\])?`)
|
||||||
|
for _, part := range strings.Split(header, ";") {
|
||||||
|
part = strings.TrimSpace(part)
|
||||||
|
matches := symbolRe.FindStringSubmatch(part)
|
||||||
|
if len(matches) > 2 {
|
||||||
|
name := matches[1]
|
||||||
|
score, _ := strconv.ParseFloat(matches[2], 64)
|
||||||
|
sym := api.RspamdSymbol{
|
||||||
|
Name: name,
|
||||||
|
Score: float32(score),
|
||||||
|
}
|
||||||
|
if len(matches) > 3 && matches[3] != "" {
|
||||||
|
params := matches[3]
|
||||||
|
sym.Params = ¶ms
|
||||||
|
}
|
||||||
|
result.Symbols[name] = sym
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// CalculateRspamdScore calculates the rspamd contribution to deliverability (0-100 scale)
|
||||||
|
func (a *RspamdAnalyzer) CalculateRspamdScore(result *api.RspamdResult) (int, string) {
|
||||||
|
if result == nil {
|
||||||
|
return 100, "" // rspamd not installed
|
||||||
|
}
|
||||||
|
|
||||||
|
threshold := result.Threshold
|
||||||
|
percentage := 100 - int(math.Round(float64(result.Score*100/(2*threshold))))
|
||||||
|
|
||||||
|
if percentage > 100 {
|
||||||
|
return 100, "A+"
|
||||||
|
} else if percentage < 0 {
|
||||||
|
return 0, "F"
|
||||||
|
}
|
||||||
|
|
||||||
|
// Linear scale between 0 and threshold
|
||||||
|
return percentage, ScoreToGrade(percentage)
|
||||||
|
}
|
||||||
|
|
@ -69,3 +69,31 @@ func ScoreToGradeKind(score int) string {
|
||||||
func ScoreToReportGrade(score int) api.ReportGrade {
|
func ScoreToReportGrade(score int) api.ReportGrade {
|
||||||
return api.ReportGrade(ScoreToGrade(score))
|
return api.ReportGrade(ScoreToGrade(score))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// gradeRank returns a numeric rank for a grade (lower = worse)
|
||||||
|
func gradeRank(grade string) int {
|
||||||
|
switch grade {
|
||||||
|
case "A+":
|
||||||
|
return 6
|
||||||
|
case "A":
|
||||||
|
return 5
|
||||||
|
case "B":
|
||||||
|
return 4
|
||||||
|
case "C":
|
||||||
|
return 3
|
||||||
|
case "D":
|
||||||
|
return 2
|
||||||
|
case "E":
|
||||||
|
return 1
|
||||||
|
default:
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// MinGrade returns the minimal (worse) grade between the two given grades
|
||||||
|
func MinGrade(a, b string) string {
|
||||||
|
if gradeRank(a) <= gradeRank(b) {
|
||||||
|
return a
|
||||||
|
}
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
|
|
||||||
|
|
@ -195,7 +195,7 @@ func (a *SpamAssassinAnalyzer) parseSpamReport(report string, result *api.SpamAs
|
||||||
// CalculateSpamAssassinScore calculates the SpamAssassin contribution to deliverability
|
// CalculateSpamAssassinScore calculates the SpamAssassin contribution to deliverability
|
||||||
func (a *SpamAssassinAnalyzer) CalculateSpamAssassinScore(result *api.SpamAssassinResult) (int, string) {
|
func (a *SpamAssassinAnalyzer) CalculateSpamAssassinScore(result *api.SpamAssassinResult) (int, string) {
|
||||||
if result == nil {
|
if result == nil {
|
||||||
return 100, "" // No spam scan results, assume good
|
return 100, "" // No spam scan results
|
||||||
}
|
}
|
||||||
|
|
||||||
// SpamAssassin score typically ranges from -10 to +20
|
// SpamAssassin score typically ranges from -10 to +20
|
||||||
|
|
|
||||||
125
web/src/lib/components/RspamdCard.svelte
Normal file
125
web/src/lib/components/RspamdCard.svelte
Normal file
|
|
@ -0,0 +1,125 @@
|
||||||
|
<script lang="ts">
|
||||||
|
import type { RspamdResult } from "$lib/api/types.gen";
|
||||||
|
import { getScoreColorClass } from "$lib/score";
|
||||||
|
import { theme } from "$lib/stores/theme";
|
||||||
|
import GradeDisplay from "./GradeDisplay.svelte";
|
||||||
|
|
||||||
|
interface Props {
|
||||||
|
rspamd: RspamdResult;
|
||||||
|
}
|
||||||
|
|
||||||
|
let { rspamd }: Props = $props();
|
||||||
|
|
||||||
|
// Derive effective action from score vs known rspamd default thresholds.
|
||||||
|
// The action header is unreliable in milter setups (always "no action").
|
||||||
|
const RSPAMD_GREYLIST_THRESHOLD = 4;
|
||||||
|
const RSPAMD_ADD_HEADER_THRESHOLD = 6;
|
||||||
|
|
||||||
|
const effectiveAction = $derived.by(() => {
|
||||||
|
const rejectThreshold = rspamd.threshold > 0 ? rspamd.threshold : 15;
|
||||||
|
if (rspamd.score >= rejectThreshold)
|
||||||
|
return { label: "Reject", cls: "bg-danger" };
|
||||||
|
if (rspamd.score >= RSPAMD_ADD_HEADER_THRESHOLD)
|
||||||
|
return { label: "Add header", cls: "bg-warning text-dark" };
|
||||||
|
if (rspamd.score >= RSPAMD_GREYLIST_THRESHOLD)
|
||||||
|
return { label: "Greylist", cls: "bg-warning text-dark" };
|
||||||
|
return { label: "No action", cls: "bg-success" };
|
||||||
|
});
|
||||||
|
</script>
|
||||||
|
|
||||||
|
<div class="card shadow-sm" id="rspamd-details">
|
||||||
|
<div class="card-header {$theme === 'light' ? 'bg-white' : 'bg-dark'}">
|
||||||
|
<h4 class="mb-0 d-flex justify-content-between align-items-center">
|
||||||
|
<span>
|
||||||
|
<i class="bi bi-shield-exclamation me-2"></i>
|
||||||
|
rspamd Analysis
|
||||||
|
</span>
|
||||||
|
<span>
|
||||||
|
{#if rspamd.deliverability_score !== undefined}
|
||||||
|
<span class="badge bg-{getScoreColorClass(rspamd.deliverability_score)}">
|
||||||
|
{rspamd.deliverability_score}%
|
||||||
|
</span>
|
||||||
|
{/if}
|
||||||
|
{#if rspamd.deliverability_grade !== undefined}
|
||||||
|
<GradeDisplay grade={rspamd.deliverability_grade} size="small" />
|
||||||
|
{/if}
|
||||||
|
</span>
|
||||||
|
</h4>
|
||||||
|
</div>
|
||||||
|
<div class="card-body">
|
||||||
|
<div class="row mb-3">
|
||||||
|
<div class="col-md-4">
|
||||||
|
<strong>Score:</strong>
|
||||||
|
<span class={rspamd.is_spam ? "text-danger" : "text-success"}>
|
||||||
|
{rspamd.score.toFixed(2)} / {rspamd.threshold.toFixed(1)}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<div class="col-md-4">
|
||||||
|
<strong>Classified as:</strong>
|
||||||
|
<span class="badge {rspamd.is_spam ? 'bg-danger' : 'bg-success'} ms-2">
|
||||||
|
{rspamd.is_spam ? "SPAM" : "HAM"}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
<div class="col-md-4">
|
||||||
|
<strong>Action:</strong>
|
||||||
|
<span class="badge {effectiveAction.cls} ms-2">
|
||||||
|
{effectiveAction.label}
|
||||||
|
</span>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{#if rspamd.symbols && Object.keys(rspamd.symbols).length > 0}
|
||||||
|
<div class="mb-3">
|
||||||
|
<div class="table-responsive mt-2">
|
||||||
|
<table class="table table-sm table-hover">
|
||||||
|
<thead>
|
||||||
|
<tr>
|
||||||
|
<th>Symbol</th>
|
||||||
|
<th class="text-end">Score</th>
|
||||||
|
<th>Parameters</th>
|
||||||
|
</tr>
|
||||||
|
</thead>
|
||||||
|
<tbody>
|
||||||
|
{#each Object.entries(rspamd.symbols).sort(([, a], [, b]) => b.score - a.score) as [symbolName, symbol]}
|
||||||
|
<tr
|
||||||
|
class={symbol.score > 0
|
||||||
|
? "table-warning"
|
||||||
|
: symbol.score < 0
|
||||||
|
? "table-success"
|
||||||
|
: ""}
|
||||||
|
>
|
||||||
|
<td class="font-monospace">{symbolName}</td>
|
||||||
|
<td class="text-end">
|
||||||
|
<span
|
||||||
|
class={symbol.score > 0
|
||||||
|
? "text-danger fw-bold"
|
||||||
|
: symbol.score < 0
|
||||||
|
? "text-success fw-bold"
|
||||||
|
: "text-muted"}
|
||||||
|
>
|
||||||
|
{symbol.score > 0 ? "+" : ""}{symbol.score.toFixed(2)}
|
||||||
|
</span>
|
||||||
|
</td>
|
||||||
|
<td class="small text-muted">{symbol.params ?? ""}</td>
|
||||||
|
</tr>
|
||||||
|
{/each}
|
||||||
|
</tbody>
|
||||||
|
</table>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{/if}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<style>
|
||||||
|
/* Darker table colors in dark mode */
|
||||||
|
:global([data-bs-theme="dark"]) .table-warning {
|
||||||
|
--bs-table-bg: rgba(255, 193, 7, 0.2);
|
||||||
|
--bs-table-border-color: rgba(255, 193, 7, 0.3);
|
||||||
|
}
|
||||||
|
|
||||||
|
:global([data-bs-theme="dark"]) .table-success {
|
||||||
|
--bs-table-bg: rgba(25, 135, 84, 0.2);
|
||||||
|
--bs-table-border-color: rgba(25, 135, 84, 0.3);
|
||||||
|
}
|
||||||
|
</style>
|
||||||
|
|
@ -6,11 +6,9 @@
|
||||||
|
|
||||||
interface Props {
|
interface Props {
|
||||||
spamassassin: SpamAssassinResult;
|
spamassassin: SpamAssassinResult;
|
||||||
spamGrade?: string;
|
|
||||||
spamScore?: number;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
let { spamassassin, spamGrade, spamScore }: Props = $props();
|
let { spamassassin }: Props = $props();
|
||||||
</script>
|
</script>
|
||||||
|
|
||||||
<div class="card shadow-sm" id="spam-details">
|
<div class="card shadow-sm" id="spam-details">
|
||||||
|
|
@ -21,13 +19,13 @@
|
||||||
SpamAssassin Analysis
|
SpamAssassin Analysis
|
||||||
</span>
|
</span>
|
||||||
<span>
|
<span>
|
||||||
{#if spamScore !== undefined}
|
{#if spamassassin.deliverability_score !== undefined}
|
||||||
<span class="badge bg-{getScoreColorClass(spamScore)}">
|
<span class="badge bg-{getScoreColorClass(spamassassin.deliverability_score)}">
|
||||||
{spamScore}%
|
{spamassassin.deliverability_score}%
|
||||||
</span>
|
</span>
|
||||||
{/if}
|
{/if}
|
||||||
{#if spamGrade !== undefined}
|
{#if spamassassin.deliverability_grade !== undefined}
|
||||||
<GradeDisplay grade={spamGrade} size="small" />
|
<GradeDisplay grade={spamassassin.deliverability_grade} size="small" />
|
||||||
{/if}
|
{/if}
|
||||||
</span>
|
</span>
|
||||||
</h4>
|
</h4>
|
||||||
|
|
|
||||||
|
|
@ -19,6 +19,7 @@ export { default as PendingState } from "./PendingState.svelte";
|
||||||
export { default as PtrForwardRecordsDisplay } from "./PtrForwardRecordsDisplay.svelte";
|
export { default as PtrForwardRecordsDisplay } from "./PtrForwardRecordsDisplay.svelte";
|
||||||
export { default as PtrRecordsDisplay } from "./PtrRecordsDisplay.svelte";
|
export { default as PtrRecordsDisplay } from "./PtrRecordsDisplay.svelte";
|
||||||
export { default as ScoreCard } from "./ScoreCard.svelte";
|
export { default as ScoreCard } from "./ScoreCard.svelte";
|
||||||
|
export { default as RspamdCard } from "./RspamdCard.svelte";
|
||||||
export { default as SpamAssassinCard } from "./SpamAssassinCard.svelte";
|
export { default as SpamAssassinCard } from "./SpamAssassinCard.svelte";
|
||||||
export { default as SpfRecordsDisplay } from "./SpfRecordsDisplay.svelte";
|
export { default as SpfRecordsDisplay } from "./SpfRecordsDisplay.svelte";
|
||||||
export { default as SummaryCard } from "./SummaryCard.svelte";
|
export { default as SummaryCard } from "./SummaryCard.svelte";
|
||||||
|
|
|
||||||
|
|
@ -12,6 +12,7 @@
|
||||||
ErrorDisplay,
|
ErrorDisplay,
|
||||||
HeaderAnalysisCard,
|
HeaderAnalysisCard,
|
||||||
PendingState,
|
PendingState,
|
||||||
|
RspamdCard,
|
||||||
ScoreCard,
|
ScoreCard,
|
||||||
SpamAssassinCard,
|
SpamAssassinCard,
|
||||||
SummaryCard,
|
SummaryCard,
|
||||||
|
|
@ -347,16 +348,19 @@
|
||||||
</div>
|
</div>
|
||||||
{/if}
|
{/if}
|
||||||
|
|
||||||
<!-- Additional Information -->
|
<!-- Spam filter analysis -->
|
||||||
{#if report.spamassassin}
|
{#if report.spamassassin || report.rspamd}
|
||||||
<div class="row mb-4" id="spam">
|
<div class="row mb-4" id="spam">
|
||||||
<div class="col-12">
|
{#if report.spamassassin}
|
||||||
<SpamAssassinCard
|
<div class={report.rspamd ? "col-lg-6 mb-4 mb-lg-0" : "col-12"}>
|
||||||
spamassassin={report.spamassassin}
|
<SpamAssassinCard spamassassin={report.spamassassin} />
|
||||||
spamGrade={report.summary?.spam_grade}
|
</div>
|
||||||
spamScore={report.summary?.spam_score}
|
{/if}
|
||||||
/>
|
{#if report.rspamd}
|
||||||
</div>
|
<div class={report.spamassassin ? "col-lg-6" : "col-12"}>
|
||||||
|
<RspamdCard rspamd={report.rspamd} />
|
||||||
|
</div>
|
||||||
|
{/if}
|
||||||
</div>
|
</div>
|
||||||
{/if}
|
{/if}
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue