I'm getting more and more dependant on it ... I've been
using,
administering,
and teaching
internet services for quite a lot of years now:
WWW,
usenet,
ftp,
e-mail,
...
Currently (2001 ...) I am using Deutsche Telekom's resp. T-Online's ADSL access. I'm going to discuss several lower and higher lever protocols:
how to transmit IP packets on ethernet, serial line, ...
the lower level of IP protocols: TCP (connection oriented), UDP (connectionless), ICMP
the user service oriented protocols
Schulferien Berlin by “Matthias Treske”
Deutsche Feiertage by “Akkordeon Spielring”
Deutsche Feiertage by “Google themselves (?!?)”
Deutsche Feiertage by “Lars Piechowiak”
Feiertage by “Mirko Prehn”
Feiertage by “Christian Maniewski”
List of XMPP client software (en.wikipedia.org)
Web_Applications@ !!! as there is some convergence between e-mailing and calendars
Webmail -- the concept of accessing your mail box through a web browser whilst enjoying ads.
SquirrelMail, free software written in php, that you are free to connect to the IMAP server of your choice
The protocol to access Hotmail is nowadays based on WebDAV, as specified in RFC2518.
Google Mail, also called gmail
I use it for splitting my incoming e-mail messages at my ISP into about 2 dozens of mail folders, which I access through IMAP mainly with squirrelmail or fetchmail,
and I also it on my Linx main computer (actually a notebook) for splitting the incoming e-mail into about a 1,000 different mail folders, which I usually access through Gnus.
Apparently there are not that many IMAP providers, that allow you to run procmail, but those that don't, usually also don't allow you to run server-side sieve. So nowadays I don't recommend procmail or sieve base server-side mail splitting.
sieve was originally meant to be a formalism to specify mail filtering rules and a mechanism to be applied within IMAP servers on incoming mail.
Nowadays you can also find it within client-side utilities, but the software may not be quite mature yet.
To be continued ...
POPFile is said to make use of a Naive Bayes classifier, which makes it different to the remainder of the crowd here, as it qualifies content on a higher level.
JHimap_utils are my own approach (written in python)
for client-side mail splitting of e-mail from the IMAP INBOX
into various other IMAP folders, as specified with the rules supplied by the user.
They originate in a time when client-side sieve-based utilites were not available yet, and that's exactly why they were developed.
To be continued ...
http://imapfilter.hellug.gr
, also client-side mail splitting,
written in Lua
http://www.showmyip.com/
:
Immediately displays your current IP address and ... .
If you're behind a NAT router, this web site helps you finding out your IP address [5], and this might be a nice starter for scripting this task, if you ever happen to need it:
curl --silent http://www.showmyip.com/ | egrep --only-matching '[0-9.]+' | head -1
... or just ...
curl http://simple.showmyip.com
On the web pages listed under the dmoz link you find quite a couple of explanations for what a Class A Address, a Class B Address, or a Class C Address is, this may be the tersest one you may find, but it's here for my own reference:
Table 1.2. IP Address Classes
address class | MSB pattern | the value of the first byte is ... | network bits | host bits |
---|---|---|---|---|
Class A | 0 | less than 128 | 8 | 24 |
Class B | 10 | from 128 to 191 | 16 | 16 |
Class C | 110 | from 192 to 223 | 24 | 8 |
(multicast) | 1110 | from 224 to 239 | ||
(reserved) | 1111 | greater than 239 |
But I like esp. the one I found in the O'Reilly book on TCP/IP Network Administration, chapter 2, named “Delivering the Data”, in the section named “The IP Address”, subsection “Address Classes”.
used by Bill Gates, Microsloth I mean.
Actually in earlier times I preferred smbmount, but recently that seems not to work any longer. so the only (reasonable) option left is smbclient.
$
cd ~/tmp/xfer #-D
is a bit like tar's--directory
, # but it's still used as leading path; #-T
makes smbclient behave a bit like tar, # thec
afterwards is tar's--create
# the letters afterwards are meant to behave like tar's operation modifiers, #g
makes smbclient only transmit files # that have the archive bit set, #a
resets that archive bit on the Win98 box # after retrieving the file from there; smbclient //hayek6/c -N -Tcga - money 2>/dev/null | tar xvf - smbclient //hayek6/c -N -Tcga - home 2>/dev/null | tar xvf - # POST: this actually also retrieved the entire directory structure from remote; # we want to get rid of empty directories: find . -depth -type d -empty -exec rmdir '{}' \; # find the files: find . -type f # ... and move them to their respective places;
$
# all the files copied won't get their original modification date/time, although the manual page says so,
# and they all also get the archive bit said, which is quite some rubbish;
# I haven't found a way to avoid all that yet;
cd ~
tar cf - money | smbclient //hayek6/c -N -Tx -
cd /export
tar cf - home/jochen_hayek/{diary,etc/{addr.s-exp,passwords}} | smbclient //hayek6/c -N -Tx -
# this will (amongst other things) reset all archive bits:
smbclient //hayek6/c -N -Tcga - home > /dev/null
The Lesson of FSP (a rather nice article on why for a specific perod of time it was important to have an alternative to FTP)
buy this book on amazon.de
If your computer's access to the Internet is not continuous, you will have to retrieve your e-mail from your mail host via an IMAP (and/or POP) capable client software. I'm using fetchmail. The author: @cite{[Eric_S_Raymond]} and his community. He presented an article on its development: The Cathedral And The Bazaar.
A protocol to access and manipulate e-mail on dedicated servers. I have written a utility in the python programming language to handle my incoming e-mail in my IMAP INBOXes (there is one INBOX per account). After moving messages from an INBOX to a dedicated mail folder, The e-mail than gets downloaded selectively using fetchmail. The expected URL: www.IMAP.org.
I personally have got 3 different usage models, that I shall try to explain together with their complications.
I think they use Cyrus,
and the mail gets sorted there into IMAP mail boxes
using procmail on the remote side -- with my own ~/.procmailrc
;
actually this is my preferred way to do mail filtering and sorting,
and I might never have worked on my own utilities,
if that method would always be applicable, but it definitely is not always applicable.
My mail doesn't stay there for longer than a few days. Usually I download all messages using fetchmail, but under certain circumstances I use Outlook or Netscape to look at the few mail boxes, that are not for mailing lists, means: the (potentially) interesting rest in some situations.
As I use (almost) the same procmail rules "there" and "here", I have no worries losing the "nice order" my messages are in "there", when I transfer them using fetchmail. Whereas I only have between 10 and 20 mail boxes there, I have maybe a hundred mail boxes in places where I keep my mail long-term, so almost the same procmail (or "whatever") rules sort mail into "just a few" mail boxes there, but into ... in the long-term mail storage places.
people / colleagues do weird things there, like oftenly sending 10 MB sized word attached documents; I hate downloading them to my notebook, as some of them can't just get deleted immediately, but I have to keep them for "quite a while". So I think the company has to "pay" for what its employees "cost" me, i.e. I keep those weird messages on the company's IMAP server[6], and obviously I don't deleted outdated e-mail regularly, that's where it actually costs the company money; but if the company doesn't teach its employees netiquette, the company has to take the consequences, and certainly not me. That IMAP server is actually and currently the HP OpenMail server, (it will most probably be MS Exchange sooner or later, as most employees are migrating), but this way or that way, that means: no chance for me to access the mail storage "genuinely" -- bad ;-(
The way I sort and filter my mail on the company IMAP server is using my own utilities written in Python, that implement an easy way of supplying mail sorting / recognition rules looking at just the mail header fields.
I access the company IMAP server of course in the company through LAN, but also as a teleworker through ISDN and sometimes "fast" modem connections.
This IMAP server only gets activated "on demand" (UWash through inetd), when I use MS Outlook to transfer single messages from one mailbox (usually) on the company IMAP server to another mailbox on my notebook IMAP server.
As soon as mail is within my Linux / UNIX environment in native UNIX flat file mail boxes, I manage it using "usual methods". These years my "usual method" here is Gnus with GNU emacs.
What do I mean by this? E.g. once in a while you would like to take a "few dozens" of "more personal" mail messages from the company mail server to your private mail storage on your notebook or so. Another example: system administrators have to transfer their users' mail from mail boxes under one IMAP server to mail boxes under another one. I currently don't know of any utility supporting that.
The organization, where I have my IMAP mailboxes[7], does not support imaps access, but instead I can ssh there and call imapd on the command line. If this is wrapped up, this is then called preauthenticated IMAP. fetchmail supports this method via a plugin.
I would actually also like to make use of this approach directly within Outlook, but I have no idea, whether this is possible at all.
My alternative approach is
to "sort of" forward stdin and stdout of a preauthenticated imapd to my local imap port,
so that Outlook on my other WinXP notebook could connect to this local imap port on my Linux notebook.
No, there's not really a security risk,
as both notebooks are protected towards the rest of the world with a router device including a firewall.
The entry in /etc/inetd.conf
looks like this:
imap stream tcp nowait root /usr/sbin/tcpd /usr/bin/ssh -p 443 -USER@HOST /usr/sbin/imapd
I even wrapped this into a little TransConnect, so that it is "firewall/proxy-tolerant".
...
The following section explains the same in german.
Actually for the time being, I only expressed it in german for the german section, as so far only german speakers have been using my non-main e-mail addresses.
Ich habe das folgende in meinem Lebend mehr als einmal geschrieben, mal freundlicher, mal weniger freundlich, deswegen schreibe ich es jetzt mal hier und verweise ab sofort auf diese Stelle und bitte dann in gewissen Fällen in einer extrem kurzen und höflich gehaltenen e-mail-Nachricht dringend darum, diese kurzen Abschnitte zu lesen und unbedingt zu berücksichtigen.
Also, ich muss das mit meinen e-mail-Adressen wohl noch mal klarstellen.
Die Kurzfassung ist:
Wenn ich nicht sage,
dass e-mail an mein <...@t-online.de>
-Adresse gehen soll,
dann soll sie immer an meine <Jochen (AT) Hayek.name>
-Adresse gehen.
Der Grund:
Die e-mail von <...@t-online.de>
kann mich echt teuer kommen,
wenn ich über Mobil-Telefon eingewählt bin,
deswegen hole ich sie (vereinfacht) nie ab,
wenn ich teuer (also z.B. per Mobil-Telefon) eingewählt bin.
In der Folge kann auch Ihre e-mail dann ganz schön lange liegen bleiben,
und wir haben ein handfestes Kommunikationsproblem.
Und deswegen ist es einfacher,
einfach nie an <...@t-online.de>
e-mail zu schicken,
außer ich bitte explizit darum.
Ich würde es eigentlich lieber vorziehen,
meine <...@t-online.de>
-e-mail-Adresse aus Ihrem Adress-Buch gestrichen zu bekommen,
weil Sie sowieso wohl fast nie benutzt werden wird.
Langfassung:
Alles was nach <Jochen (AT) Hayek.name>
geht
(und glauben Sie mir, bitte, da gehen jeden Tag hunderte von e-mails hin!!)
wird zweistufig durch Filterregeln sortiert,
an denen ich seit langer Zeit arbeite
und die mir garantieren,
dass ich vom dem ganzen "spam"[8]
nur wirklich minimal behelligt werde.
Ich habe dazu Filterregeln bei meinem "Mailbox-Provider" quasi "hinter <Jochen (AT) Hayek.name>
"
(und das ist nicht T-Online),
welche gewissermaßen nach nochmal besonderen Regeln vorsortieren.
Die garantieren mir z.B. insbesondere auch, dass ich nichts über (z.B.) Mobil-Telefon herunterlade, was größer als 15kb ist.
Wenn ich nämlich über Mobil-Telefon eingewählt bin, zahle ich nämlich für jedes einzelne Kilo-Byte und zwar nicht zu wenig.
Und wenn die Disziplin nicht mehr funktioniert, dass mein e-mail-Verkehr grundsätzlich über meine Haupt-e-mail-Adresse abläuft, dann bleibt mir nichts anderes übrig, als die e-mail-Adressen, die mir wegen Nicht-Beachtung dieser Disziplin zu teuer werden, abzumelden.
Und das ist selbstverständlich keine Maßnahme gegen Sie persönlich sondern gegen spam etc.
See `IMAP'! POP is the `older' version of IMAP, to explain it in very simple words.
There is much to say about SMTP, but currently I only write down here, what I need to handle my e-mail forwarding on ACM.org (a VMS host);
$ mail MAIL> set forward in%"""...@@...""" MAIL> show forward Your mail is being forwarded to IN%"...@@...". MAIL> set noforward
Used between news servers and also between a news server and a news client.
NNTPSERVER=sunsite.auc.dk NNTPSERVER=news.cs.tu-berlin.de NNTPSERVER=news.fu-berlin.de NNTPSERVER=news.uni-stuttgart.de NNTPSERVER=news.lexware.de NNTPSERVER=news.t-online.de NNTPSERVER=news1.shuttle.de NNTPSERVER=news.microsoft.com # these do not want to talk to me: NNTPSERVER=news.informatik.hu-berlin.de
$
testhost $NNTPSERVER -a -T 300 > active-$NNTPSERVER-UNSORTED # give it much longer to time out, because it's so busy ...
# maybe the result of $(date '+%s') instead of 999 : cat /var/lib/news/history ~jo_hayek/.nnmail-cache | perl -ne 'm/^<(.*)>/ && print "<$1>\n"' | sort -u | perl -ne 'chop; print $_,"\t999~\n"' > /var/lib/news/history-NEW # SuSE's suck just uses flat file history, so this is not worth while at all; # hence I don't need inn's makehistory at all, and hence no fake inn installation: /usr/lib/news/bin/makehistory -or # IN: /var/lib/news/history; OUT: /var/lib/news/history.{dir,pag}
$
cd /home/jochen_hayek/NNTP-suck
# maybe you want to add:
# --cleanup -> sucknewsrc, sucknewsrc.old, suck.newsrc
suck ${NNTPSERVER} --postfix -${NNTPSERVER} --cleanup > NEWSBOX-${NNTPSERVER}
suck ${NNTPSERVER} --postfix -${NNTPSERVER} > NEWSBOX-${NNTPSERVER}
# $(perl -ane 'print "--ng $F[0] \n"' sucknewsrc-TEMPLATE )
JHparse_news_batch.py --verbose \
$(sed 's/^\([^ ]*\).*/--ng \1/' sucknewsrc-${NNTPSERVER}) \
NEWSBOX-${NNTPSERVER}
# POST: leaves the articles in subdir ./tmp
cd tmp; \
for i in *
do echo "$i":
cat "$i" >> ~/procmail-dir/"$i" && rm "$i"
done
##########
suck ${NNTPSERVER} --postfix -${NNTPSERVER} --timeout 300 --dir_msgs ${NNTPSERVER} --batch_lmove lmove.cf-${NNTPSERVER}
$
perl -ane 'print $F[0],"\n"' sucknewsrc-*.?? | sort -u | sed 's/$/ 0/' > sucknewsrc-TEMPLATE
for i in sucknewsrc-*.??
do : echo $i:
perl -ae '
while(<>) { @F=split; $high{$F[0]} = $F[1]; };
foreach(sort keys %high) { print "$_ $high{$_}\n" }
' sucknewsrc-TEMPLATE "$i" > "$i-NEW"
cmp -s "$i" "$i-NEW" || echo "*** $i: needs updating"
done
rename 's/-NEW$//' sucknewsrc-*.??-NEW
$
suck "$1" -bl /var/lib/news/active-"$1" -c
# in exactly this order, because the 2nd pass does the clean up:
find ..... -type f | while read i; do fgrep -il base64 $i && munpack "$i" && rm -f "$i"; done
find ..... -type f | while read i; do echo "$i":; uudecode "$i"; rm -f "$i"; done
SLIP came before PPP.
AFAIR the original SLIP software required a root installation on the remote side.
Once the Linux kernel and the SuSE Linux distribution supported that, Internet from at home started really being as few hassle as possible.
Used by ADSL providers, e.g. by Deutsche Telekom for their product T-DSL.
...
This method has been used for quiet a while within SuSE Linux
by the utilities wvdial, kppp, smpppd, ...,
as they have their own places for keeping PPP accounts and passwords.
Not that this is safer than keeping that in pap-secrets
or chap-secrets
,
but at least accounts and passwords are only kept in that one place preferred by the respective utility.
This is seriously one nice piece of software.
I am always intrigued to search in this area,
if I forget where the persistence option
gets set;
but it is in /etc/ppp/peers/pppoe
:
##idle # MUST NOT BE USED # this is the "anti-idle" etc. option persist # this is for reconnecting 60 seconds after a connection got interrupted: holdoff 60
Instead of the original SLIP software you can also use SLiRP, which does not require root privileges and keeps remaining useful in the context of PPTP VPN-s, as it is pretty easy to set up.
Keep in mind, that pppd can also be started on the command line on the remote side by non-privileged users, and that if you succeed talking to it using a local pppd, there is no need for authentication. Why should there be any such need? You are already an authenticated visitor.
Having said all this, how do we make our local PPP software talk the remote slirp or pppd?
connect to a shell prompt on the remote side
launch slirp
make your local PPP software speak to the remote PPP software, i.e. slirp
if your local system is Linux, the previous was already done through a chat script, which in turn was launched by pppd
if your local system is Windows, you launch your slirp resp. pppd in a Post Dial Terminal Screen, which in turn is launched by a Dial-Up Connection attempt (see this lenghty description SLiRP setup for Windows 98!)
enjoy your TCP/IP connections
dmoz@
Section 3, named “Private Address Space” lists three blocks of the IP address space for private networks:
10.0.0.0 | - | 10.255.255.255.255 | 24-bit block | a single class A network |
172.16.0.0 | - | 172.31.255.255 | 20-bit block | 16 contiguous class B network numbers |
192.168.0.0 | - | 192.168.255.255.255 | 16-bit block | 255 contiguous class C network numbers |
I found a nice article titling “Appendix: SOCKS -- A Basic Firewall” including a section on “How to SOCKSify a client” and a section on “What SOCKS expects”.
I am not quite sure, whether these instructions apply to all or most socks resp. socks5 implementations or just to QNX.
From the document's abstract:
This document specifies a protocol for encapsulation of an arbitrary network layer protocol over another arbitrary network layer protocol.
Another IP protocol, on my system listed in /usr/include/netinet/in.h
.
This protocol is one of the alternative basises for VPN-s.
Section 8, named “Acknowledgements”, says:
Links to suppliers:
This router has two different options for the WAN port:
a real ordinary WAN with e.g. PPPoE
"just" a LAN; and this device may act as a DHCP client itself on this WAN port
In either case it does NAT-routing, firewalling, port forwarding, ...
I attached that device to a corporate network, because it offers me
NAT-routing (so I can attach more than one computer, even if got only hold on a single network plug),
a (local) DHCP server on the LAN ports side,
and a DHCP client on the WAN port side.
Alright, alright, having a firewall on the WAN port providing the local LAN clients with some privacy towards the corporate LAN hosts is quite useful.
First I had trouble attaching the WAN port to that big corporate LAN, as I didn't figure immediately, that the WAN port only provides a 10-MBit line, whereas that corporate LAN provided a just 100-MBit line and they didn't play together. The (temporary) solution was to have a dual-speed hub in between.
Actually my seeking Netgear's support on that occasion is quite a story to tell:
When I tried to sort the above issue out with their U.S. support on the phone, after quite a while of waiting in the telephone queue and a short while discussing the problem the guy on the phone explained me, that kind of problem wasn't covered by their free support policy, but I could go for paid support.
When I posted a support query and also when I sent them an e-mail, the guys tried to drag me to the (free) telephone support.
The german support guy I first got on the telephone, promised me a call back, but he never did. But when I had him on the phone, at least he ensured me, they themselves in Munich used the device pretty much that way, although w/o a DCHP server to be launched a request to from the WAN port. Knowing that I tried that configuration at home w/o and w/ DHCP server. So I somehow knew something extremely odd was going on there.
The german support guy I got on my second telephone call, told me to insert a network hub between the router the corporate LAN (and run an Ethernet sniffer on a sidebranch on that hub) in order to find out, where the traffic (signalled by the ACT LED) comes from. That actually led me to putting a dual-speed hub before the WAN-port.
I still (?) have trouble connecting to my ISP through an external ADSL modem -- TBD.
IMO this device entirely acts like a FVS318 with an integrated WG602 (a 802.11g access point), accept for the WAN port, that can also communicate at 100 Mbits/second.
It can do web filtering. It does report web accesses to an e-mail address, that the admin can define. The reported lines look like this:
[ALLOW:www.netvenda.com] Source:10.0.4.200 Wednesday, 06 Aug 2003 14:14:15
And this is the one-liner, that shows me which IP addresses on my LAN accessed web sites outside:
grep --only-matching ' Source:[0-9\.]*' ~/Mail/system.admin.firewall | sort -u
Also this device acts like a FVS318, but with extra alternative WAN port, where you can attach something with a DB-9 connector understanding AT modem commands, be it an ordinary analogous modem or an ISDN adapter.
The serial line is thought of to act as a fallback solution, in case the ADSL resp. cable modem temporarily doesn't provide you with a proper connection.
But in the case of a family member of mine, the serial line serves as the standard line. Employing this device may sound like a bit of overkill, but this way, he surfs at least firewall protected. And also he will almost not be able to entirely misconfigure his Internet connection setup (what we assume he did once -- with “disastrous” impact). This device has a nice little reset button, and that will do what we expect it to do. And reconfiguring the device should be a piece of cake, as the device allows saving his settings to a disk file and restoring them back again. So if we burn that settings file on a CD-ROM, what on earth can go wrong there??
Netgear, Germany, in contrary to Netgear, U.S., explain, that this device only supports ISDN adapters on the DB-9.
This device integrates a DSL modem into something like the FVS318.
I have trouble connecting to my ISP through T-DSL.
I promise, I read the troubleshooting section of the reference manual on the resource CD.
But only when I called Netgear's german service hotline, I happenend to hear for the first time of UR2-standard-conformance. Apparently Deutsche Telekom requests part suppliers to conform with their UR2-standard, but their own "old" T-DSL port installations do not conform with UR2 themselves.
But this nice little device here requests conformance with UR2, so I have to get my T-DSL port upgraded. Crucial question: will my old Telekom-supplied ADSL-modem still work with the upgrade port? The Telekom technician in charge gave me a quick feed back, after setting up UR2 on my port, and he confirmed, that the old modem would also and still do its job with UR2. I did not want to be forced to no longer be able to use that modem, and therefore I prefer to preserve compatibility for as long as possible. This nice little device here is only just another option. (A (NAT, ...) router with an integrated WLAN access point (like the WGR614) after this stand-alone modem is yet another option.)
The other settings parameters were not a problem at all: "LLC", VPI=>1, VCI=>32, and I also already upgraded the firmware to the latest release supplied by Netgear.
TBD ...
on the Open Directory Project itself
[5] e.g. for registering with a DynDNS service, esp. one that your nice NAT router doesn't cope with ...
[6] when I'm very interested in the message itself w/o the attached document, I detach that document, maybe after storing it in my tmp or transient directory
[7] they are apparently using the U-Wash imapd
[8] das ist der unerbetene e-mail-Schrott