Saving large attachments to memory

[MikeJohnson]

Subject: Saving large attachments to memory

Please forgive the cross-posting.

My add-in saves several 200MB attachments to a memory buffer.

I notice that the calls to IStream::Read
can take wildly variable amounts of time,
for identically-sized attachments.

IStream::Read consistently takes over two minutes(!) to return for the
first two attachments,
but it returns in only a few seconds for the subsequent attachments!

It would make sense to me if only the first call to IStream::Read took
a very long time,
as the operating system may need to actually commit memory.
But, the delay is occurring for the first *and* second large
attachments (but none thereafter)!

Please find my code below.


Would anyone have any thoughts as to what might be causing the
performance difference?

Are there any properties of the attachments that could explain the
difference?

Is there a different way to copy the attachment data to memory?

Thanks very much for any help or recommendations!

Mike


static BOOL SaveAttachmentFromMemoryUsingExMAPI (IAttach* pAttach)
{
/*
* Try to get the contents of the attachment.
*/

BYTE* pAttachmentBuffer = NULL;
ULONG ulAttachmentSize = 0L;

IStream* pStream = NULL;

/*
* Fetch an IStream interface to the attachment's
PR_ATTACH_DATA_BIN property.
*/
if ((hr = pAttach->OpenProperty (PR_ATTACH_DATA_BIN, &IID_IStream,
STGM_READ, NULL, (LPUNKNOWN*) &pStream)) != S_OK)
{
Error();
}

if (pStream == NULL)
{
Error();
}


/*
* Fetch the attachment's size.
*/
STATSTG statstg;
memset (&statstg, 0, sizeof(STATSTG));

if ((hr = pStream->Stat (&statstg, STATFLAG_NONAME)) != S_OK)
{
Error();
}

// Bail if attachment size >= 4GB!

ulAttachmentSize = statstg.cbSize.LowPart;


/*
* Get memory buffer to copy attachment into.
* Note: GetBigMemoryBuffer uses C RTL realloc() to obtain a memory
buffer.
*/
pAttachmentBuffer = GetBigMemoryBuffer (ulAttachmentSize);


/*
* Read attachment contents into buffer from stream.
*/
ULONG ulBytesToRead = ulAttachmentSize;
ULONG ulBytesRead = 0L;

while (ulBytesToRead > 0)
{
if ((hr = pStream->Read (pAttachmentBuffer, ulBytesToRead,
&ulBytesRead)) != S_OK)
{
Error();
}
ulBytesToRead -= ulBytesRead;
}

if (pStream != NULL)
{
pStream->Release();
pStream = NULL;
}

/*
* Save memory buffer containing attachment data.
*/
SaveMemoryBufferContainingAttachmentData();
}

[Dan Mitchell]

Post by MikeJohnson
IStream::Read consistently takes over two minutes(!) to return for the
first two attachments,
but it returns in only a few seconds for the subsequent attachments!

Have you tried watching network traffic (just with task manager or
whatever)? I'm just guessing, but I'd suspect that once you start reading
attachments MAPI begins pulling the whole lot over the network to your
local PC, and it's finished doing that by the time you've read the first
two attachments; the following ones go faster because they're already on
your local PC somehow.

(another way to test this would be to compare two messages with (say)
1000mb of attachments; one with 5x200mb, and one with 10x100mb
attachments. If I'm right about this, then the first one will speed up
after the first two (maybe) attachments, the second one will speed up
after the first four -- and if you change it to be 5x100mb, it'll speed up
after the first one).

-- dan