This project is read-only.

Performance when downloading mail with attachment

Topics: Issues
Nov 6, 2013 at 3:58 PM
Hi,

when downloading a Message with one 800kb attachment, the download took ~10 minutes and cpu usage of the process on a 8 core machine was about 15% constantly.
message.Download(ImapX.Enums.MessageFetchMode.Full);
It seems the time is spent in Base64.cs in public static byte[] FromBase64(string s).
Is there a chance to optimize this? I would like to use the api in a server side processing of tons of mails, lots of them with attachment ..

And by the way, your api is great! I find it very intuitive :-)

Ciao
Thorsten
Nov 6, 2013 at 4:32 PM
I put a breakpoint in Base64.FromBase64(string s) and it looks like the method is beeing called ever and ever again with the same input string growing in length.
Maybe adding the data chunks to ContentStream, each time raise a property changed what forces a base64 decoding of the actual string?

ImapX.dll!ImapX.EncodingHelpers.Base64.FromBase64(string s) Line 90
ImapX.dll!ImapX.Attachment.ContentOnPropertyChanged(object sender, System.ComponentModel.PropertyChangedEventArgs args) Line 61 + 0x49 bytes
ImapX.dll!ImapX.MessageContent.OnPropertyChanged(string name) Line 256 + 0x59 bytes
ImapX.dll!ImapX.MessageContent.ContentStream.set(string value) Line 101 + 0x17 bytes
ImapX.dll!ImapX.MessageContent.ProcessCommandResult(string data) Line 218 + 0x96 bytes
ImapX.dll!ImapX.ImapBase.SendAndReceive(string command, ref System.Collections.Generic.IList<string> data, ImapX.Parsing.CommandProcessor processor, System.Text.Encoding encoding) Line 351 + 0x1f bytes
ImapX.dll!ImapX.MessageContent.Download() Line 237 + 0xe0 bytes
ImapX.dll!ImapX.Message.Download(ImapX.Enums.MessageFetchMode mode, bool reloadHeaders) Line 533 + 0x12 bytes
ImapX2Test.exe!ImapX2Test.MyApp.AsyncGetMails() Line 54 + 0x18 bytes

Ciao
Thorsten
Coordinator
Nov 6, 2013 at 4:41 PM
Hi TProelss,

yes, this happens for attachments, as the decoding is bound to the property change. I will fix this.

Thanks,

Pavel
Coordinator
Nov 27, 2013 at 10:46 PM
Hi Thorsten,

i fixed the issue you found! Please check the latest code in the repository.

Kind regards,

Pavel
Nov 28, 2013 at 11:03 PM
Edited Nov 28, 2013 at 11:05 PM
Hi Pavel,

I have tried to download a message using MessageFetchMode.Full but it does not seem to be resolved with the repository downloaded today.

For example I tried downloading an email about 10Mb in size, but after one hour it is still downloading, and this is on server hardware.

Kind regards,
Kristof
Coordinator
Nov 29, 2013 at 9:04 AM
Hi Kristof,

i will make some tests today and see if can find a solution! Thank you!

Best regards,

Pavel
Coordinator
Dec 6, 2013 at 9:19 PM
Hi Kristof,

the low performance was caused by a great number of string operations and also regular expressions. The code in the repository contains updated and fixed code, please take a look.

Greets,

Pavel