Image attachment breaks parser

Topics: Issues
Feb 16, 2014 at 3:26 AM
Hi Pavel,

as I mentioned in previous message, I was processing a lot of image attachments for software for processing puzzle hunt email and one user's images were consistently triggering this problem when iterating over attachments. If it's not obvious what the problem is, I can try to fish out the offending email and send it to you. It downloads ok, and I can get all the email metadata but once I start processing the image attachments, I get this traceback. I'm pretty sure the email was legit because other clients (e.g. gmail) could view the email perfectly,

Darren
p.s. the download raw method worked beautifully.

The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.

at System.Convert.FromBase64_Decode(Char* startInputPtr, Int32 inputLength, Byte* startDestPtr, Int32 destLength)
at System.Convert.FromBase64CharPtr(Char* inputPtr, Int32 inputLength)
at System.Convert.FromBase64CharArray(Char[] inArray, Int32 offset, Int32 length)
at System.Security.Cryptography.FromBase64Transform.TransformBlock(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount, Byte[] outputBuffer, Int32 outputOffset)
at ImapX.EncodingHelpers.Base64.FromBase64(String s) in c:\proj\DotNet\imapx-34274\ImapX\EncodingHelpers\Base64.cs:line 105
at ImapX.Attachment.get_FileData() in c:\proj\DotNet\imapx-34274\ImapX\Attachment.cs:line 47
at Download.imapMain(String connString, Int32 N) in C:\proj\TSP\tspdownload\download.fs:line 257
Feb 16, 2014 at 6:33 PM
Edited Feb 16, 2014 at 6:35 PM
I dug into this a bit more and I have an example document that breaks ImapX if you'd like a copy. Taking a look at the exception

Image

I suspect the problem is the leading space character in this buffer,

Darren
inputBytes
inputBytes
{byte[34]}
    [0]: 32
    [1]: 66
    [2]: 79
    [3]: 68
    [4]: 89
    [5]: 91
    [6]: 49
    [7]: 46
    [8]: 50
    [9]: 93
    [10]: 32
    [11]: 78
    [12]: 73
    [13]: 76
    [14]: 32
    [15]: 66
    [16]: 79
    [17]: 68
    [18]: 89
    [19]: 91
    [20]: 49
    [21]: 46
    [22]: 50
    [23]: 46
    [24]: 77
    [25]: 73
    [26]: 77
    [27]: 69
    [28]: 93
    [29]: 32
    [30]: 78
    [31]: 73
    [32]: 76
    [33]: 41
Feb 16, 2014 at 6:44 PM
One final piece of the puzzle, that line doesn't seem to be base64 image encoding, it's actually the text
BODY[1.2]NILBODY[1.2.MIME]NIL)
which doesn't appear in the actual email, - looks like an IMAP instruction that's getting into the image parsing.

Darren
Coordinator
Feb 19, 2014 at 6:39 AM
Hi Darren,

thank you for the great research work! The main problem here is that ImapX doesn't read literal strings taking in account their size which is provided by the server. This is the reason why sometimes the content part body contains additional characters which causes the decoder to fail with an exception.

I'm working on an update, hope to release it soon.

Greets,

Pavel
Coordinator
Apr 8, 2014 at 7:19 PM
Good evening!

Please have a look at the latest code in the repository. Various fixes have been done. I'd be happy if you can test it and give some feedback!

Best regards,

Pavel
Apr 17, 2014 at 7:40 PM
Hi Pavel, sorry I haven't replied more quickly. Very excited to check this out, should have time later this week, and I can beat it up thoroughly on a large test set, DArren On Tue, Apr 08, 2014 at 12:19:09PM -0700, pavel_azanov wrote:
Apr 21, 2014 at 7:38 AM
On Tue, Apr 08, 2014 at 12:19:09PM -0700, pavel_azanov wrote:
Apr 21, 2014 at 7:38 AM
Ok, just tried to build current repository and there was one file not found: Error 3 Source file 'Enums\IdleState.cs' could not be found C:\proj\DotNet\imapx-35473\ImapX\CSC ImapX Looks like it's not in the repo, Darren On Tue, Apr 08, 2014 at 12:19:09PM
-0700, pavel_azanov wrote:
Coordinator
Apr 21, 2014 at 1:50 PM
Hi Darren,

i'm implementing idle support, and forgot to include the IdleState.cs file last time. I made another commit now, it's in the repository now.

Greets,

Pavel
Apr 23, 2014 at 3:19 PM
On Mon, Apr 21, 2014 at 06:50:11AM -0700, pavel_azanov wrote:
May 19, 2014 at 6:58 PM
Hi Pavel, I just built 35999 and it still chokes on this particular piece of email. Built nicely. Is there anything else I can do to help debug? My C# isn't spectacular but maybe I could step through the code and try to narrow down a little better why
it's getting confused about what section it's in, Darren On Mon, Apr 21, 2014 at 06:50:11AM -0700, pavel_azanov wrote:
Coordinator
May 21, 2014 at 8:48 AM
Hi Darren,

I will release another fix on Friday, would be kind if you can test it!

Best regards,

Pavel
Aug 28, 2014 at 9:50 PM
Hi Pavel

I use the latest version, and still get the same error "The input is not a valid Base-64 string as it contains a non-base 64 character, more than two padding characters, or an illegal character among the padding characters.

at System.Convert.FromBase64_Decode(Char* startInputPtr, Int32 inputLength, Byte* startDestPtr, Int32 destLength)
at System.Convert.FromBase64CharPtr(Char* inputPtr, Int32 inputLength)
at System.Convert.FromBase64CharArray(Char[] inArray, Int32 offset, Int32 length)
at System.Security.Cryptography.FromBase64Transform.TransformBlock(Byte[] inputBuffer, Int32 inputOffset, Int32 inputCount, Byte[] outputBuffer, Int32 outputOffset)
at ImapX.EncodingHelpers.Base64.FromBase64(String s)
at ImapX.EncodingHelpers.StringDecoder.DecodeBase64(String value, Encoding encoding)
at ImapX.MessageBody.get_Html()

Please help

Thanks
Coordinator
Sep 1, 2014 at 10:01 AM
Hi Vnwind,

thank you for your message, I will try to isolate this error, and would be great if you can provide me with a log file as described in the documentation. This will help me to simulate the described situation.

Greets,

Pavel
Sep 15, 2014 at 8:48 PM
Hi Pavel

The problem is I can't replicate on dev enviroment, I tried to use the problem email in dev environment but it works fine. So I can't get the debug log file of the problem.
Is there anything else I can try? The error only happens on live server.

Thanks
Sep 15, 2014 at 8:52 PM
I'm still struggling with the same problem. I might be able to try and replicate but problem is it's served up by google and I can't easily give you access to that account. If I can replicate the problem with a different local imap server I might be able
to get you a test account. Darren On Mon, Sep 15, 2014 at 01:48:53PM -0700, Vnwind wrote: