Message unique identification?

Topics: Help requests
Mar 4 at 8:23 AM
Hello, I try to write e-mail archive utility. So I walk thrue a loop, where I get UID and Internal Date.
I though it should be unique - so I make filename from them and check if such file exists.
If so I don't download message again.

But now I see repeated messages in my archive folder, where internalDate and content are same, but UID is different.

How can I identify already downloaded messages uniquely?

Thank you
Miroslav
Coordinator
Mar 4 at 12:11 PM
Hi Miroslav,

to identify a message uniquely, please use the Message.MessageId property.

If you made any changes to the ImapClient.Behavior.RequestedHeaders property, make sure that it contains MessageHeader.MessageId or is null.

Greets,

Pavel
Mar 4 at 3:07 PM
Hi Pavel,
in my case MessageId is null.
How should I do message.download for fill this item?

Additional question - can I do query to message list e.g. not older than 10 days?

Sincerely Miroslav
Mar 4 at 3:38 PM
Hi Pavel,
I get null messageID in case download Full:
mx.Download(Enums.MessageFetchMode.Full, True)
Dim pom As String = mx.MessageId
Greetings Miroslav
Coordinator
Mar 11 at 9:41 AM
Hi Miroslav,

please check if the client.Behavior.RequestedHeaders contains "Message-Id". It should be requested fine.

Greets,

Pavel
Mar 23 at 5:30 PM
Hi Pavel,
sorry for late answer - I was abroad.

You didn't answer my first question: What is the minimal Enums.MessageFetchMode for getting MessageID?

It is important for me not to get large collection for first reguest.

Thank you very much.

Miroslav
Mar 24 at 1:48 PM
Edited Mar 24 at 4:43 PM
Hi Pavel,
I have Message-ID empty.
Client.Behavior.RequestedHeaders does not contain Message-ID item
There is Fetchmode Basic even though I use Full.

Thank you for your help. Miroslav
Coordinator
Mar 24 at 4:41 PM
Hi Miroslav,

I will post a sample here a bit later for you.

Greets,

Pavel
Coordinator
Mar 25 at 5:03 PM
So, here the promised example:
' Initializing the client
Dim client = New ImapClient("imap.gmail.com", True)

' Configuring the client to only download the headers
client.Behavior.MessageFetchMode = MessageFetchMode.Headers

' Setting the headers to be fetched (only Message-Id)
client.Behavior.RequestedHeaders = New String() {MessageHeader.MessageId}

If client.Connect() Then

    If client.Login("name@gmail.com", "password") Then

        ' Requesting messages
        Dim messages = client.Folders.Inbox.Search()

        ' .. Filter the messages by Message.MessageId
        Dim filteredMessages = messages.Where(Function(msg As Message)
                                                  Return msg.MessageId = "Sample-Message-Id"
                                              End Function)

        ' After you filtered the messages, reset the requested headers by
        client.Behavior.RequestedHeaders = Nothing

        ' Or client.Behavior.RequestedHeaders = MessageHeaderSets.Minimal;

        For Each message As Message In filteredMessages

            ' Download the messages with another fetch mode. 
            ' The second parameter indicates that the headers should be requested one more time
            message.Download(MessageFetchMode.Basic, True)

        Next

    End If

End If
If you have more questions, feel free to ask.

Best regards,

Pavel
Mar 25 at 6:07 PM
Edited Mar 25 at 6:10 PM
Hi Pavel, I am very sorry, but both MessageId and MessageUid are empty.

imapClient.Behavior.MessageFetchMode = Enums.MessageFetchMode.Headers
imapClient.Behavior.RequestedHeaders = {"MessageHeader.MessageId", "MessageHeader.MessageUid"}
Dim messages = imapClient.Folders.Inbox.Search("SINCE 20-Mar-2014")


Client.Behavior.RequestedHeaders contains both requested items.

If it could help, we can estabilish TeamViewer session.

Sincerely Miroslav
Coordinator
Mar 27 at 2:01 PM
Hi Miroslav,

looking at your code:
quot;MessageHeader.MessageId", "MessageHeader.MessageUid"
you sholuld remove the quotes. But we can also try solving it in a TeamViewer session as soon as I'm back home. You can add me on Skype: pavel_azanov

Greets,

Pavel
Mar 28 at 10:13 AM
Hi Pavel,

removing quotas helped - now I got the values.
But unfortunately messageId is not much usefull for file name creating.

So I plan to return to UID.
May you confirm me, that UID is definitely unique and constant and cannot change e.g. by server restart or by deleting some messages?

Thank you very much.
Miroslav
Coordinator
Mar 28 at 11:13 AM
Edited Mar 28 at 11:13 AM
Hi Miroslav,

the message UID is not a very reliable value for message identification. According to the specification of IMAP, it SHOULD not change between sessions, however, should not doesn't mean all servers will preserve same UIDs between sessions.

What's the problem in using the MessageId for filename creation? If it contains characters vbeing invalid in path, you can simply hash the value and use the hash for the filename.

Greets,

Pavel
Mar 28 at 11:24 AM
Hi Pavel,

thank you for very usefull information.
Using hash of messageID seems to be great idea.
I will try it.

Miroslav
Mar 31 at 11:05 PM
Be aware that multiple messages can share the same Message-Id value (this tends to happen when you are subscribed to a mailing-list where you received a copy of the message from the list and directly via a Cc or something).

Also, there is a security issue with using the Message-Id as a unique ID for a message in that if an attacker knows the Message-Id of a message that he wants to block, he can send you a new message with the same Message-Id value. Not super likely, but it is just something to be aware of.

I'm not sure what exactly to use for your purposes (and the Message-Id may be fine for your use case, at least for personal use). Perhaps a hash of the most recent Received header? Assuming you trust your mail server, the most recent Received header will be "unique" in that it is time-stamped as well as containing the source hostname and IP of the sender. It might be vulnerable to an attack sent from the same IP during that 1 second period, but otherwise I'd imagine it to be fairly "safe".
Coordinator
Apr 1 at 8:25 AM
Hi Jeffrey,

thank you for the information!

Using the Received header can be a good option and will work best in most cases. This makes me think of adding a new property to the Message object, which will contain a unique identificator (hash) of it, based on message-id and the received headers.

Greets,

Pavel
Apr 3 at 8:47 AM
Hi Pavel,

please, let us know when unique identificator will be available.

Thank you
Miroslav
Coordinator
Apr 3 at 10:29 PM
Hi Miroslav,

I will push the latest code to the repository on weekend, will make an update for the bodystrcutureparser as well as fix the issue about junk data in downloaded attachments. The unique identificator update will also be included.

Greets,

Pavel