Pop3mail

Pop3 client to download email (including attachments) from the inbox via the commandline or Elixir API.
Written in Elixir, using an Erlang pop3 client with SSL support derived from the epop package.
Decodes multipart content, quoted-printables, base64 and encoded-words.

Implemented RFC's in Pop3mail to decode email

• RFC 5322 previously RFC 822 and RFC 2822
• RFC 2045
• RFC 2046
• RFC 2047
• RFC 2231
• RFC 2557

Before you start

• This program reads from a POP3 mail server, which means that it can only download mail from the inbox folder. If you want to access other folders you will need an IMAP client.
• Handling big attachments requires some processing memory. Normally the program needs about 30Mb RAM (for the whole OS process), but to process an email with attachments it temporary needs 3 times of the total size of the email attachments as additional memory.
• Elixir programmers can replace the default Pop3mail.Base64Decoder with their own.
• On linux when there is not enough memory, the program will end as 'Killed.' It's killed by the OOM Killer. Run dmesg to see the log message.
• On windows when there is not enough memory the program get stuck, or worse windows get stuck.
• Do NOT run the script as root.
• Downloaded attachments can contain viruses, addware or malicious scripts.
• This program does NOT convert charsets and neither does it add a BOM. If a message is send in ISO-8859-1, CP1251, KOI8-R it wil be stored as such. Sometimes you must change the locale/charset/encoding (LC_CTYPE luit, chcp) in your terminal/device/program to be able to read the content. Elixir programmers can use codepagex to perform conversions to utf-8.

Gmail users:

• Whether the read mail is permanently deleted or not, depends on your Gmail settings, and not on the delete parameter of this program.
• Gmail returns chunks of maximum 250-350 emails. Repeatedly run this program to get all emails.

Installation from scratch

Install Elixir

Follow the instructions on http://elixir-lang.org/install.html

Also install git and optionally also rebar3.

Erlang/OTP version

The error 'connection_failed' is thrown with OTP 26 and Erlang 1.14. The pop3client must be upgraded to verify the server as mentioned in gen_smtp issue. For now, use OTP 25. To revert back to OTP 25 and Erlang 1.13:

$ erl -eval 'erlang:display(erlang:system_info(otp_release)), halt().'  -noshell
$ cat /usr/lib/erlang/releases/RELEASES
$ sudo apt-get remove erlang
$ wget https://packages.erlang-solutions.com/erlang-solutions_1.0_all.deb
$ curl --no-progress-meter https://www.debian.org/releases/ | grep Debian | grep '<q>'
$ # use codename bullseye to get OTP 25 in the following command:
$ sudo dpkg -i erlang-solutions_1.0_all.deb
$ sudo apt-get update
$ apt-cache policy esl-erlang
$ # if erlang-solutions was already installed, change codename in this file: 
$ sudo vi /etc/apt/sources.list.d/erlang-solutions.list 
$ sudo apt update
$ apt-cache policy esl-erlang
$ sudo apt-get install esl-erlang
$ cat /usr/lib/erlang/releases/RELEASES
$ asdf current
$ asdf list-all elixir | grep 25
$ asdf install elixir 1.15.6-otp-25
$ asdf global elixir 1.15.6-otp-25
$ asdf current
$ cd pop3mail
$ mix deps.clean --all
$ mix clean
$ mix deps.get
$ mix compile

Clone project

$ git clone https://github.com/nico-amsterdam/pop3mail.git

Compile & unit test

$ cd pop3mail
$ mix deps.get
$ mix test

For usage, see usage chapter below.

Install in an Elixir project

1. Add pop3mail to your list of dependencies in mix.exs:

        def deps do
          [{:pop3mail, "~> 1.4"}]
        end

Upgrade instructions 1.3.1 to 1.3.2

Run:

mix deps.clean erlpop
mix deps.update pop3mail

After the mix deps.clean erlpop command, the deps/erlpop directory should be gone, and also the _build/dev/lib/erlpop and _build/test/lib/erlpop should be vanished.

Upgrade instructions 1.3.0 to 1.3.1

Version 1.3.1 doesn't require erlpop as github dependency anymore, because it is now available in hex.pm as 'pop3client' and added as dependency for pop3mail. Remove {:erlpop, github: "nico-amsterdam/erlpop"} in your mix.exs. If you don't mix reports: 'Dependencies have diverged'

Usage

Commandline script

The script downloads email and writes the content in the inbox folder.

$ pop3mail_downloader --help
$ pop3mail_downloader --username=<your email username> --password=<your email password> --max=10 --raw

or without shell/batch script:

*nix

$ mix run_pop3mail --help
$ mix run_pop3mail --username='<your email username>' --password='<your email password>' --max=10 --raw

Windows

C:\pop3mail\mix run_pop3mail --help
C:\pop3mail\mix run_pop3mail --username="<your email username>" --password="<your email password>" --max=10 --raw

The script defaults to Gmail, but you can specify other POP3 server and port settings.

Use in Elixir

Documentation is available online

Example:

$ iex -S mix

# notice that you must use single quotes here
iex(1)> {:ok, client} = :epop_client.connect('user@gmail.com', 'password', 
...(1)>   [:ssl, {:addr, 'pop.gmail.com'}, {:port, 995}, {:user, 'user@gmail.com'}])
iex(2)> :epop_client.stat(client) 
iex(3)> {:ok, mail_content} = :epop_client.bin_retrieve(client, 1) 
iex(4)> {:message, header_list, body_content } = :epop_message.bin_parse(mail_content)
iex(5)> Pop3mail.header_lookup(header_list, "Subject")
iex(6)> Pop3mail.header_lookup(header_list, "From")
iex(7)> Pop3mail.header_lookup(header_list, "Date")
iex(8)> part_list = Pop3mail.decode_body_content(header_list, body_content)
iex(9)> length(part_list)
iex(10)> part = Enum.at(part_list, 0)
iex(11)> part.media_type
iex(12)> part.filename
iex(13)> part.charset
iex(14)> part.content
iex(15)> :epop_client.delete(client, 1)
iex(16)> {:ok, mail_content} = :epop_client.bin_retrieve(client, 2) 
iex(17)> {:message, header_list, body_content } = :epop_message.bin_parse(mail_content)
iex(18)> Pop3mail.header_lookup(header_list, "Subject")
iex(19)> :epop_client.quit(client)

Spam folder

You better turn off the spam folder of your email account if you don't want to miss any email with this program. In Gmail you cannot turn it off, but you can create a filter for spam with the option 'Never send it to spam'.

Reset Gmail

Gmail remembers which mails are already read. Fortunetely Gmail can be reset to re-read all emails.

Login in www.gmail.com. In Gmail webmail Settings > Forwarding and POP/IMAP, select another option for POP, like Download mail from now on. Save change. Go back to settings and select Download all mail, Save change.

Now your email client should download all mail again.

Access Gmail with less secure apps

Google only trusts google apps. Gmail is trusted, but pop3mail not. You will notice that authentication fails, and google will sent you a security warning by email. Access with less secure apps can be turned on for your google account at: https://myaccount.google.com/lesssecureapps 2 step verification must be on: https://myaccount.google.com/signinoptions/two-step-verification In the 2 step verification section, it is possible to set an app password: https://myaccount.google.com/apppasswords Use the 16 character password.

Google unlock captcha

If you get an error 'web login required', push the 'Continue' button in the browser:

https://accounts.google.com/DisplayUnlockCaptcha

License

MIT

Acknowledgment

Thanks Erik Søe Sørensen for upgrading the Epop client to the OTP 15 Erlang version.