2026-10-31 Send missing net news articles
=========================================

I'm running a little net news network of retro-tech and role-playing
games for some friends. There's a web interface if you're interested
in taking a look.

The three servers are:

* campaignwiki.org run by me

* grenzland.club run by lkh

* wilderland.ovh run by kyonshi

Both lkh and kyonshi also have other new servers connected.

Each server also has some groups. campaignwiki.games is for our
games, for example, and grenzland.club is for the administration of
the whole network including all the other services we run.

When setting up the groups, however, I made a mistake: I had
misconfigured my net news server so that it wouldn't send articles
from the grenzland.club group to the grenzland.club server.

I didn't notice for the longest time. For almost all newsgroups, the
articles were exchanged correctly. For the grenzland.club group,
articles only propagated one way. I could still post! And if people
checked my news server, they'd see the replies. But if people checked
any other news server, they would not.

So step 1 was to fix my configuration of outgoing messages. The
entries for the respective peers in /etc/news/newsfeeds now read:

    # kyonshi
    wilderland/wilderland.ovh\
           :!*,campaignwiki.*,erebor.*,grenzland.*,\
            rec.games.frp*,de.rec.spiele/!local\
           :Af,Ap,C20,G12,U5,<32768,Tm\
           :innfeed!
    
    # lkh
    grenzland/grenzland.club\
           :!*,campaignwiki.*,erebor.*,grenzland.*,\
            rec.games.frp*,de.rec.spiele/!local\
           :Af,Ap,C20,G12,U5,<32768,Tm\
           :innfeed!

And of course I had to reload this:

    sudo -u news ctlinnd reload newsfeeds "Fix groups for grenzland network"

But now I wanted to send the missing articles to the other servers.
How does this happen? I read a lot of man pages and talked to
@alexlehm@mastodon.social on the #netnews channel on tilde.chat. And
this is what I have.

First, find the list of files with a message ID indicating that they
originated on my server, and save the filenames in a batch file:

    grep -il '^Message-ID: .*campaignwiki' /var/spool/news/articles/grenzland/club/* > batch

And then attempt to post them, with debug output:

    /usr/lib/news/bin/innxmit -d grenzland.club /root/batch

It turns out that innxmit does not use the peering information in
/etc/news/innfeed.conf. There, I map grenzland to grenzland.club
and provide a username and password. What was required was an entry in
/etc/news/passwd.nntp instead.

With that done, the innxmit command above worked as intended.

And the beauty of it is that the receiving server checks for
duplicates. At least I hope that's why it ended up rejecting five of
the eight articles. This is what it looked like:

    < 200 grenzland.club InterNetNews server INN 2.7.3 ready (transit mode)
    >MODE STREAM
    < 203 Streaming permitted
    > CHECK <10nfu9o$bdhb$1@sibirocobombus.campaignwiki>
    > CHECK <10rvjkb$u6ro$1@paraelectrobombus.campaignwiki>
    > CHECK <115g4ap$216nb$1@paraelectrobombus.campaignwiki>
    > CHECK <116qd83$305fd$1@paraelectrobombus.campaignwiki>
    > CHECK <116qf9r$305fd$2@paraelectrobombus.campaignwiki>
    > CHECK <11bdrsk$f3l6$1@paraelectrobombus.campaignwiki>
    > CHECK <11bdtcf$f911$1@paraelectrobombus.campaignwiki>
    > CHECK <11bnne9$1jf65$1@paraelectrobombus.campaignwiki>
    < 238 <10nfu9o$bdhb$1@sibirocobombus.campaignwiki> Send it
    > TAKETHIS <10nfu9o$bdhb$1@sibirocobombus.campaignwiki>
    > [ article 1242 ]
    > .
    < 238 <10rvjkb$u6ro$1@paraelectrobombus.campaignwiki> Send it
    > TAKETHIS <10rvjkb$u6ro$1@paraelectrobombus.campaignwiki>
    > [ article 967 ]
    > .
    < 238 <115g4ap$216nb$1@paraelectrobombus.campaignwiki> Send it
    > TAKETHIS <115g4ap$216nb$1@paraelectrobombus.campaignwiki>
    > [ article 1476 ]
    > .
    < 238 <116qd83$305fd$1@paraelectrobombus.campaignwiki> Send it
    > TAKETHIS <116qd83$305fd$1@paraelectrobombus.campaignwiki>
    > [ article 2607 ]
    > .
    < 238 <116qf9r$305fd$2@paraelectrobombus.campaignwiki> Send it
    > TAKETHIS <116qf9r$305fd$2@paraelectrobombus.campaignwiki>
    > [ article 2728 ]
    > .
    < 238 <11bdrsk$f3l6$1@paraelectrobombus.campaignwiki> Send it
    > TAKETHIS <11bdrsk$f3l6$1@paraelectrobombus.campaignwiki>
    > [ article 1698 ]
    > .
    < 238 <11bdtcf$f911$1@paraelectrobombus.campaignwiki> Send it
    > TAKETHIS <11bdtcf$f911$1@paraelectrobombus.campaignwiki>
    > [ article 3050 ]
    > .
    < 238 <11bnne9$1jf65$1@paraelectrobombus.campaignwiki> Send it
    > TAKETHIS <11bnne9$1jf65$1@paraelectrobombus.campaignwiki>
    > [ article 2752 ]
    > .
    < 439 <10nfu9o$bdhb$1@sibirocobombus.campaignwiki>
    < 439 <10rvjkb$u6ro$1@paraelectrobombus.campaignwiki>
    < 439 <115g4ap$216nb$1@paraelectrobombus.campaignwiki>
    < 439 <116qd83$305fd$1@paraelectrobombus.campaignwiki>
    < 439 <116qf9r$305fd$2@paraelectrobombus.campaignwiki>
    < 239 <11bdrsk$f3l6$1@paraelectrobombus.campaignwiki>
    < 239 <11bdtcf$f911$1@paraelectrobombus.campaignwiki>
    < 239 <11bnne9$1jf65$1@paraelectrobombus.campaignwiki>

From RFC 2980:

* 238 no such article found, please send it to me

* 239 article transferred ok

* 439 article transfer failed

#News #NNTP

2026-11-01. Talking with xwindows on the #netnews channel got me
some more insights.

First, compare the Message-IDs on both servers.

One one server:

    grep ^Message-ID: /var/spool/news/articles/grenzland/club/* \
    | cut -d\  -f2 > campaignwiki-msg-ids

On the other server:

    grep ^Message-ID: /var/spool/news/articles/grenzland/club/* \
    | cut -d\  -f2 > grenzmark-msg-ids

Transfer the file from one server to the other:

    rsync -ai grenzland:grenzland-msg-ids .
    rsync -ai grenzland-msg-ids paraelectrobombus:

Compute the diff, using fish:

    diff (sort grenzland-msg-ids | psub) (sort campaignwiki-msg-ids | psub)

The five messages that did not get sent are in the diff!

And incidentally, I'm also missing a lot of the remote messages. One
thing at a time, however.

Messages that are too old will be rejected by a server. Messages that
have been rejected before will be in the cache of rejected messages.
This presents difficulties.

This is what I tried.

On the target server, edit /etc/news/inn.conf and set the following:

* artcutoff: 0 (was 10)

* remembertrash: false (was true)

Restart the server:

    ctlinnd xexec innd

Edit /etc/news/inn/expire.ctl and set the following:

* `/remember/:0 (was 11)

With artcutoff 0, I should be able to send missing articles that are
too old, but it won't work since it was already rejected.

Now I was getting 438 "Duplicate" errors. And indeed, if I checked one
of the Message-IDs I wanted to transmit on the target server with
grephistory, I got back /dev/null. That is, the article is known but
not stored anywhere. It's in the history file, as a rejection.

The only option seems to be to wait for 11 days until the original
artcutoff has passed, and try again. Setting remembertrash and
`/remember/ might not be necessary.

2026-11-01. Talking with xwindows on the #netnews channel got me
some more insights.

First, compare the Message-IDs on both servers.

One one server:

    grep ^Message-ID: /var/spool/news/articles/grenzland/club/* \
    | cut -d\  -f2 > campaignwiki-msg-ids

On the other server:

    grep ^Message-ID: /var/spool/news/articles/grenzland/club/* \
    | cut -d\  -f2 > grenzmark-msg-ids

Transfer the file from one server to the other:

    rsync -ai grenzland:grenzland-msg-ids .
    rsync -ai grenzland-msg-ids paraelectrobombus:

Compute the diff, using fish:

    diff (sort grenzland-msg-ids | psub) (sort campaignwiki-msg-ids | psub)

The five messages that did not get sent are in the diff!

And incidentally, I'm also missing a lot of the remote messages. One
thing at a time, however.

Messages that are too old will be rejected by a server. Messages that
have been rejected before will be in the cache of rejected messages.
This presents difficulties.

This is what I tried.

On the target server, edit /etc/news/inn.conf and set the following:

* artcutoff: 0 (was 10)

* remembertrash: false (was true)

Restart the server:

    ctlinnd xexec innd

Edit /etc/news/inn/expire.ctl and set the following:

* /remember/: 0 (was 11)

With artcutoff 0, I should be able to send missing articles that are
too old, but it won't work since it was already rejected.

Now I was getting 438 "Duplicate" errors. And indeed, if I checked one
of the Message-IDs I wanted to transmit on the target server with
grephistory, I got back /dev/null. That is, the article is known but
not stored anywhere. It's in the history file, as a rejection.

The only option seems to be to wait for 11 days until the original
artcutoff has passed, and try again. Setting remembertrash and
/remember/ might not be necessary.

Going in the other direction, however! I only set artcutoff to 0 and
send the missing articles from Grenzland to Campaign Wiki. The biggest
problem was that the diff had produced a list of Message-IDs separated
by CR NL instead of just NL. That took my way too long to figure out
and grephistory kept reporting errors. 😅

But then, finally, it was done:

    < 438 <8ed951da-91b9-4f73-b724-fda8c64c833e@app.fastmail.com> Duplicate
    < 438 <11bdrsk$f3l6$1@paraelectrobombus.campaignwiki> Duplicate
    < 438 <11bdtcf$f911$1@paraelectrobombus.campaignwiki> Duplicate
    < 438 <11bnne9$1jf65$1@paraelectrobombus.campaignwiki> Duplicate
    < 239 <10k7lu7$jduh$1@grenzland.club>
    < 239 <10kab49$qs9b$1@grenzland.club>
    < 239 <10kabb5$qsuo$1@grenzland.club>
    < 239 <10kac5n$qtj9$1@grenzland.club>
    < 239 <10kalop$r5ap$1@grenzland.club>
    < 239 <10kano0$r726$1@grenzland.club>
    < 239 <10kbdsc$rpkd$1@grenzland.club>
    < 239 <10kbnd4$s0pt$1@grenzland.club>
    < 239 <10kbts4$s6ge$1@grenzland.club>
    < 239 <10kd4ps$t3j1$1@grenzland.club>
    < 239 <10kdjtm$teau$1@grenzland.club>
    < 239 <10l8rjk$3se9t$1@grenzland.club>
    < 239 <10l9m91$3t9ld$1@grenzland.club>
    < 239 <10lqfv3$iffl$1@grenzland.club>
    < 239 <10m238q$rq3n$1@grenzland.club>
    < 239 <10ma7e7$163gi$1@grenzland.club>
    < 239 <10mf5us$1c62b$1@grenzland.club>
    < 239 <10mg63q$1ddlc$1@grenzland.club>
    < 439 <780158B4-EF25-45C3-94D9-D11AAD0DCFD0@wilderland.ovh>
    < 439 <A9094241-5CA8-4249-BAAE-F516BAAE1D35@wilderland.ovh>
    < 439 <F9A49DC3-63CD-4389-93C1-C863C178D24A@wilderland.ovh>
    < 439 <1EA26F40-8E3B-4B49-BB12-312BD4F90743@wilderland.ovh>
    < 239 <10qi581$26lu4$1@grenzland.club>
    < 239 <10qiai6$26s89$1@grenzland.club>
    < 439 <1B12F1D7-76E9-4679-80E2-D33A01F7D5A0@wilderland.ovh>
    < 439 <1369A55F-A1FD-4897-8EC4-24F60C5DEECF@wilderland.ovh>

Again:

* 239 article transferred ok

* 438 already have it, please don't send it to me

* 439 article transfer failed

One last potential reason for refusal, according to xwindows is that a
server will refuse messages that predate the local creation date of
the channel. Then again, suspiciously, all the rejected messages are
from wilderland.ovh.

Checking /var/log/news/news I find six error messages with matching
Message-IDs saying "439 HTML Multipart". I think this was one user
from Wilderland replying via the mail gateway.

Going in the other direction, however! I only set artcutoff to 0 and
send the missing articles from Grenzland to Campaign Wiki. The biggest
problem was that the diff had produced a list of Message-IDs separated
by CR NL instead of just NL. That took my way too long to figure out
and grephistory kept reporting errors. 😅

But then, finally, it was done:

    < 438 <8ed951da-91b9-4f73-b724-fda8c64c833e@app.fastmail.com> Duplicate
    < 438 <11bdrsk$f3l6$1@paraelectrobombus.campaignwiki> Duplicate
    < 438 <11bdtcf$f911$1@paraelectrobombus.campaignwiki> Duplicate
    < 438 <11bnne9$1jf65$1@paraelectrobombus.campaignwiki> Duplicate
    < 239 <10k7lu7$jduh$1@grenzland.club>
    < 239 <10kab49$qs9b$1@grenzland.club>
    < 239 <10kabb5$qsuo$1@grenzland.club>
    < 239 <10kac5n$qtj9$1@grenzland.club>
    < 239 <10kalop$r5ap$1@grenzland.club>
    < 239 <10kano0$r726$1@grenzland.club>
    < 239 <10kbdsc$rpkd$1@grenzland.club>
    < 239 <10kbnd4$s0pt$1@grenzland.club>
    < 239 <10kbts4$s6ge$1@grenzland.club>
    < 239 <10kd4ps$t3j1$1@grenzland.club>
    < 239 <10kdjtm$teau$1@grenzland.club>
    < 239 <10l8rjk$3se9t$1@grenzland.club>
    < 239 <10l9m91$3t9ld$1@grenzland.club>
    < 239 <10lqfv3$iffl$1@grenzland.club>
    < 239 <10m238q$rq3n$1@grenzland.club>
    < 239 <10ma7e7$163gi$1@grenzland.club>
    < 239 <10mf5us$1c62b$1@grenzland.club>
    < 239 <10mg63q$1ddlc$1@grenzland.club>
    < 439 <780158B4-EF25-45C3-94D9-D11AAD0DCFD0@wilderland.ovh>
    < 439 <A9094241-5CA8-4249-BAAE-F516BAAE1D35@wilderland.ovh>
    < 439 <F9A49DC3-63CD-4389-93C1-C863C178D24A@wilderland.ovh>
    < 439 <1EA26F40-8E3B-4B49-BB12-312BD4F90743@wilderland.ovh>
    < 239 <10qi581$26lu4$1@grenzland.club>
    < 239 <10qiai6$26s89$1@grenzland.club>
    < 439 <1B12F1D7-76E9-4679-80E2-D33A01F7D5A0@wilderland.ovh>
    < 439 <1369A55F-A1FD-4897-8EC4-24F60C5DEECF@wilderland.ovh>

Again:

* 239 article transferred ok

* 438 already have it, please don't send it to me

* 439 article transfer failed

One last potential reason for refusal, according to xwindows is that a
server will refuse messages that predate the local creation date of
the channel. Then again, suspiciously, all the rejected messages are
from wilderland.ovh.

Checking /var/log/news/news I find six error messages with matching
Message-IDs saying "439 HTML Multipart". I think this was one user
from Wilderland replying via the mail gateway.