Until two weeks ago I was using an IMAP server (based on Zimbra) for my work email, but the date for migration to Gmail arrived with no choice to postpone…

I was very tied to using my current setup, where:

  • offlineimap was downloading all the email to a local maildir folder,
  • imapfilter classified the email into folders based on local options
  • mutt accessed the maildir folder for working with the emails
  • a script to remove duplicate emails from disk imapdedup.py before next sync

But, with the change and peculiarities for Gmail, it was no longer working… I was trying several times with different combinations of folder translations but each one took approximately one day to sync all emails, to find out the next issue with the folder translation.

After this, I tried to just skip using my old setup and switch to web interface… I was already using it for personal email for years, but with lot of mailing lists and lots of emails, didn’t worked the way I wanted.

I finally went the intermediate way: using imapfilter directly against Gmail and customize a bit the inbox folders.

This allowed me to adapt the filtering, so that I can still have a readable INBOX with all the relevant mails and keep at the same time the mails sorted as I used to.

My inbox is now configured as multiple inbox with the following filters:

  • is:starred
  • (is:unread AND NOT label:Tags/_pending)

Those show below my regular INBOX folder, allowing to still feature some emails for tracking, and all the others filtered into folders that are not in the pending sort are also displayed.

This is powered with some filters (Gmail side), that keep my inbox clean:

  • to:(-myemail1 -myemail2) -> Skip Inbox, Apply label "Tags/_pending"
  • to:(myemail or myemail2) (invite.ics OR invite.vcs) has:attachment -> Delete it
  • from:([email protected]) -> Skip Inbox, Apply label "Tags/_pending"

Now, the imapfilter part via this .imapfilter/config.lua file:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
---------------
--  Options  --
---------------

options.timeout = 60
options.subscribe = true
options.create = true
options.expunge = true

----------------
--  Accounts  --
----------------

-- Connects to "imap1.mail.server", as user "user1" with "secret1" as
-- password.
MAILSERVER = IMAP {
    server = 'imap.gmail.com',
    username = 'MYEMAIL',
    password = 'MYAPPPASSWORD',
    ssl = "tls1"
}
-- My email
myuser = 'MYUSERINEMAILS'

function mine(messages)
    email=messages:contain_cc(myuser)+messages:contain_to(myuser)+messages:contain_from(myuser)
    return email
end

function filter(messages,email,destination)
    messages:contain_from(email):move_messages(destination)
    messages:contain_to(email):move_messages(destination)
    messages:contain_cc(email):move_messages(destination)
    messages:contain_field('sender', email):move_messages(destination)
    messages:contain_field('List-ID', email):move_messages(destination)
end

function deleteold(messages,days)
    todelete=messages:is_older(days)-mine(messages)
    todelete:move_messages(MAILSERVER['[Gmail]/Papelera'])
end

function deleteoldcases(messages,days)
    todelete=messages:is_older(days)
    todelete:move_messages(MAILSERVER['[Gmail]/Papelera'])
end

function markread(messages)
    toread=messages:select_all()
    toread:mark_seen()
end

-- Define the msgs we're going to work on

-- Move sent messages to INBOX to later sorting
-- sent = MAILSERVER['sent']:select_all()
-- sent:move_messages(MAILSERVER['INBOX'])

INBOX = MAILSERVER['INBOX']:select_all()
pending = MAILSERVER['Tags/_pending']:select_all()
todos =  pending + INBOX

todos:contain_subject('Undelivered Mail Returned to Sender'):move_messages(MAILSERVER['[Gmail]/Papelera'])
todos:contain_subject('[sbr-stack] Report: OSP - Stale cases: '):move_messages(MAILSERVER['[Gmail]/Papelera'])
todos:contain_subject('[ sbr-stack-emea ] Cron <[email protected]'):contain_from('(cron Daemon)'):move_messages(MAILSERVER['[Gmail]/Papelera'])
todos:contain_from('[email protected]'):move_messages(MAILSERVER['[Gmail]/Papelera'])

todos:contain_subject('Supplier Remittance Advice- Autogenerated please do not respond to this mail'):move_messages(MAILSERVER['Tags/Privado/Gastos'])

filter(todos:is_seen(),'[email protected]',MAILSERVER['Tags/Lists/OpenStack/gerrit'])

-- Mark as read messages sent from my user
markread(todos:contain_from(myuser))

-- Delete google calendar forwards
todos:contain_to('[email protected]'):move_messages(MAILSERVER['[Gmail]/Papelera'])

filter(todos:contain_subject('[PNT] '),'[email protected]',MAILSERVER['[Gmail]/Papelera'])

-- Filter CPG
filter(todos:contain_subject('Red Hat - Group '),'[email protected]',MAILSERVER['Tags/WORK/Customers/CPG'])

-- Delete messages about New accounts created
usercreated=todos:contain_subject('New Red Hat user account created')*todos:contain_from('[email protected]')
usercreated:move_messages(MAILSERVER['[Gmail]/Papelera'])

-- Search messages from CPG's
cpg = MAILSERVER['Tags/WORK/Customers/CPG']:select_all()
cpg:contain_subject('Red Hat - Group  - '):move_messages(MAILSERVER['[Gmail]/Papelera'])
cpg:contain_subject('(Unpublished)'):move_messages(MAILSERVER['[Gmail]/Papelera'])
cpg:contain_subject(': Where is '):move_messages(MAILSERVER['[Gmail]/Papelera'])

-- Move bugzilla messages
filter(todos:contain_subject('] New:'),'[email protected]',MAILSERVER['Tags/WORK/_bugzilla/new'])
filter(todos,'[email protected]',MAILSERVER['Tags/WORK/_bugzilla'])
bz = MAILSERVER['Tags/WORK/_bugzilla']:select_all()

-- Move unseen requests or answers
reqans=bz:contain_subject('needinfo requested:'):is_unseen() + bz:contain_subject('needinfo canceled:'):is_unseen()
reqans:move_messages(MAILSERVER['Tags/WORK/_bugzilla/reqans'])

reqans=MAILSERVER['Tags/WORK/_bugzilla/reqans']:is_seen()
reqans:move_messages(MAILSERVER['Tags/WORK/_bugzilla'])

-- Clasify on product
bz:contain_field('X-Bugzilla-Product', 'Red Hat Customer Portal'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/portal'])
bz:contain_field('X-Bugzilla-Product', 'Red Hat Enterprise Linux'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhel'])
bz:contain_field('X-Bugzilla-Product', 'Red Hat Satellite'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhn'])
bz:contain_field('X-Bugzilla-Product', 'Red Hat Enterprise Virtualization Manager'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhev'])
bz:contain_field('X-Bugzilla-Product', 'Fedora'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/fedora'])
bz:contain_field('X-Bugzilla-Product', 'Red Hat Ceph Storage'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/ceph'])
bz:contain_field('X-Bugzilla-Product', 'Red Hat Gluster Storage'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/gluster'])

bz:contain_field('X-Bugzilla-Product', 'Red Hat OpenStack'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhosp'])
bz = MAILSERVER['Tags/WORK/_bugzilla/rhosp']:select_all()

-- Clasify on component for OSP
bz:contain_field('X-Bugzilla-Component', 'openstack-ceilometer'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhosp/ceilometer'])
bz:contain_field('X-Bugzilla-Component', 'openstack-cinder'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhosp/cinder'])
bz:contain_field('X-Bugzilla-Component', 'openstack-designate'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhosp/designate'])
bz:contain_field('X-Bugzilla-Component', 'openstack-glance'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhosp/glance'])
bz:contain_field('X-Bugzilla-Component', 'openstack-heat'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhosp/heat'])
bz:contain_field('X-Bugzilla-Component', 'openstack-ironic'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhosp/ironic'])
bz:contain_field('X-Bugzilla-Component', 'openstack-keystone'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhosp/keystone'])
bz:contain_field('X-Bugzilla-Component', 'openstack-neutron'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhosp/neutron'])
bz:contain_field('X-Bugzilla-Component', 'openstack-nova'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhosp/nova'])
bz:contain_field('X-Bugzilla-Component', 'rhel-osp-director'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhosp/director'])
bz:contain_field('X-Bugzilla-Component', 'rhosp-director'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhosp/director'])

bz = MAILSERVER['Tags/WORK/_bugzilla/rhev']:select_all()
bz:contain_field('X-Bugzilla-Component', 'vdsm'):move_messages(MAILSERVER['Tags/WORK/_bugzilla/rhev/vdsm'])

-- Move support messages once read into the Customer/cases folder
filter(todos:contain_subject('Case '):is_seen(),'[email protected]',MAILSERVER['Tags/WORK/Customers/cases'])

MAILSERVER['[Gmail]/Papelera']:select_all():mark_seen()

support = MAILSERVER['Tags/WORK/Customers/cases']:select_all()
-- Restart the search only for messages in Other to also process if we have new rules

support:contain_subject('is about to breach its SLA'):move_messages(MAILSERVER['[Gmail]/Papelera'])
support:contain_subject('has breached its SLA'):move_messages(MAILSERVER['[Gmail]/Papelera'])
support:contain_subject(' has had no activity in '):move_messages(MAILSERVER['[Gmail]/Papelera'])
markread(support:contain_subject('(WoC)'))
markread(support:contain_subject('(Closed)'))

-- Only work on already read messages
support = MAILSERVER['Tags/WORK/Customers/cases']:select_all():is_seen()

-- Process all remaining messages in INBOX + all read messages in pending-sort for mailing lists and move to lists folder
notminelistas=todos-mine(todos)
notminelistas:contain_field('List-ID','<'):move_messages(MAILSERVER['Tags/Lists'])
filter(notminelistas,'list', MAILSERVER['Tags/Lists'])
filter(todos:is_seen(),'list', MAILSERVER['Tags/Lists'])
filter(todos:is_seen(),'googlegroups.com', MAILSERVER['Tags/Lists'])
filter(todos,'bounces',MAILSERVER['Tags/Lists'])

-- Add RH lists, INBOX and _pending and Fedora default bin for reprocessing in case a new list has been added
lists = todos:is_seen() + MAILSERVER['Tags/Lists']:select_all() + MAILSERVER['Tags/Lists/Fedora']:select_all()

todos:contain_subject('unsubcribe'):move_messages(MAILSERVER['[Gmail]/Papelera'])
lists:contain_subject('unsubcribe'):move_messages(MAILSERVER['[Gmail]/Papelera'])

-- Mailing lists
filter(INBOX:is_seen(),'[email protected]',MAILSERVER['Tags/Lists/WORK/Tech/coderepos'])
filter(lists,'[email protected]',MAILSERVER['Tags/Lists/WORK/SysEng/CNV/kubevirt-dev'])
filter(lists,'[email protected]',MAILSERVER['Tags/Lists/WORK/SysEng/CNV/metalkube-dev'])

-- Fedora
filter(lists,'kickstart-list',MAILSERVER['Tags/Lists/Fedora/kickstart'])
filter(lists,'[email protected]',MAILSERVER['Tags/Lists/Fedora/Ambassador'])
filter(lists,'[email protected]',MAILSERVER['Tags/Lists/Fedora/infra'])
filter(lists,'[email protected]',MAILSERVER['Tags/Lists/Fedora/announce'])
filter(lists,'lists.fedoraproject.org',MAILSERVER['Tags/Lists/Fedora'])

-- OSP
filter(lists,'[email protected]',MAILSERVER['Tags/Lists/OpenStack'])
filter(lists,'[email protected]',MAILSERVER['Tags/Lists/OpenStack/Operators'])
filter(lists,'[email protected]',MAILSERVER['Tags/Lists/OpenStack/es'])
filter(lists,'[email protected]',MAILSERVER['Tags/Lists/OpenStack/rdo'])

filter(lists,'@bugs.launchpad.net',MAILSERVER['Tags/Lists/OpenStack/launchpad'])
lists:contain_field('X-Launchpad-Notification-Type', 'bug'):move_messages(MAILSERVER['Tags/Lists/OpenStack/launchpad'])

filter(lists,'[email protected]',MAILSERVER['Tags/Lists/Others/pgsql-hackers'])
filter(lists,'[email protected]',MAILSERVER['Tags/Lists/Others/pgsql-hackers'])

-- Filter  messages not filtered back to INBOX
pending:move_messages(MAILSERVER['INBOX'])

-- Start processing of messages older than:
maxage=365

-- Delete old messages from mailing lists

maxage=180
deleteold(MAILSERVER['Tags/Lists'],maxage)
deleteold(MAILSERVER['Tags/Lists/Fedora'],maxage)
deleteold(MAILSERVER['Tags/Lists/Fedora/Ambassador'],maxage)
deleteold(MAILSERVER['Tags/Lists/Fedora/announce'],maxage)
deleteold(MAILSERVER['Tags/Lists/Fedora/infra'],maxage)
deleteold(MAILSERVER['Tags/Lists/Fedora/kickstart'],maxage)
deleteold(MAILSERVER['Tags/Lists/OpenStack'],maxage)
deleteold(MAILSERVER['Tags/Lists/OpenStack/es'],maxage)
deleteold(MAILSERVER['Tags/Lists/Others/pgsql-hackers'],maxage)
deleteold(MAILSERVER['Tags/Lists/WORK/SysEng/CNV/kubevirt-dev'],maxage)
deleteold(MAILSERVER['Tags/Lists/WORK/SysEng/CNV/metalkube-dev'],maxage)



-- Delete old BZ tickets
maxage=30
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/rhosp'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/rhosp/ceilometer'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/rhosp/cinder'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/rhosp/designate'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/rhosp/glance'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/rhosp/heat'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/rhosp/ironic'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/rhosp/keystone'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/rhosp/neutron'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/rhosp/nova'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/ceph'],maxage)

maxage=30
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/portal'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/rhel'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/rhn'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/fedora'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/gluster'],maxage)
deleteoldcases(MAILSERVER['Tags/WORK/_bugzilla/security'],maxage)


-- delete old cases
maxage=30

-- for each in $(cat .imapfilter/config.lua|grep -i cases|tr " ,()" "\n"|grep cases|sort|uniq|grep -v ":" );do echo "deleteoldcases($each,maxage)";done
deleteoldcases(MAILSERVER['Tags/WORK/Customers/cases'],maxage)

-- Empty trash every 7 days
maxage=7
old=MAILSERVER['[Gmail]/Papelera']:is_older(maxage)
old:move_messages(MAILSERVER['[Gmail]/Papelera'])

I think that is more or less self-explanatory, but in short, first sorts mails into folders based on the sender, recipient, keywords and finally, applies expiration policy to the folders to remove old emails that might not be relevant anymore.

In this case, it also keeps messages in which I was directly involved (removing, via the function to delete messages that are in the variable ‘mine’)

During the next days I’ll try to get back email in mutt, but this at least makes the usage in the meantime more bearable…

Enjoy!

Like this post? Share on: TwitterFacebookEmail


Pablo Iranzo Gómez Avatar Pablo Iranzo Gómez opensource enthusiast and Lego fan doing some python simple programs like @redken_bot in telegram, etc
Comments

comments powered by Disqus

Keep Reading


Reading Time

~5 min read

Published

Category

tech

Tags

Stay in Touch