Troubleshooting ASP.Net memory usage issues?

I'm responsible for a pair of IIS 6 webservers that run our production
ASP.Net application. We push code on a monthly basis, and about two ws
after our October code push, we started to run into occasional webserver
errors.
The application pool is configured to run as a domain user which has
permissions to the web code. When we have issues, the webserver will return
the ASP.Net "Server Application Unavailable" page, and our app pool memory
usage is really high (anywhere from 2 to 4 times our normal usage). Non
asp.net content is served up just fine. If I recycle the application pool
or do an iisreset, memory usage drops back to normal and the site starts
working again.
Nothing is logged to the eventlog or http error log, and the only errors we
see from our application has been the occasional "Exception of type
System.OutOfMemoryException was thrown." The same page doesn't error every
time, and the function I've seen throw exceptions is incredibly generic and
used heavily throughout our entire application (or so our devs have told
me). Our application doesn't even log an error every time this has
happened.
How should I go about troubleshooting this? Our devs are saying it's a
server issue, but I'm leaning towards a code issue. These servers are
running Windows 2003 Server standard, fully updated as of this last wend.What session management are you using. InProc sessions are major memory
consumers.
Eliyahu Goldin,
Software Developer
Microsoft MVP [ASP.NET]
http://msmvps.com/blogs/egoldin
http://usableasp.net
"Daniel Peterson" <pythas@.hotmail.com> wrote in message
news:CB009C9F-B38F-4937-8AF9-256502E90CDE@.microsoft.com...
> I'm responsible for a pair of IIS 6 webservers that run our production
> ASP.Net application. We push code on a monthly basis, and about two ws
> after our October code push, we started to run into occasional webserver
> errors.
> The application pool is configured to run as a domain user which has
> permissions to the web code. When we have issues, the webserver will
> return the ASP.Net "Server Application Unavailable" page, and our app pool
> memory usage is really high (anywhere from 2 to 4 times our normal usage).
> Non asp.net content is served up just fine. If I recycle the application
> pool or do an iisreset, memory usage drops back to normal and the site
> starts working again.
> Nothing is logged to the eventlog or http error log, and the only errors
> we see from our application has been the occasional "Exception of type
> System.OutOfMemoryException was thrown." The same page doesn't error
> every time, and the function I've seen throw exceptions is incredibly
> generic and used heavily throughout our entire application (or so our devs
> have told me). Our application doesn't even log an error every time this
> has happened.
> How should I go about troubleshooting this? Our devs are saying it's a
> server issue, but I'm leaning towards a code issue. These servers are
> running Windows 2003 Server standard, fully updated as of this last
> wend.
Hi Daniel,
It's a code issue. It will be one of 2 possible reasons: 1. Memory Leak.
Yes, .Net applications can leak memory, if they use unmanaged resources that
are not disposed properly. 2. Memory Fragmentation. If the same block of
memory is re-used as, for example, an application-level collection that has
members added and removed over a long period of time, that memory can become
fragmented, necessitating the allocation of more memory to create contiguous
blocks when they are needed.
HTH,
Kevin Spencer
Chicken Salad Surgeon
Microsoft MVP
"Daniel Peterson" <pythas@.hotmail.com> wrote in message
news:CB009C9F-B38F-4937-8AF9-256502E90CDE@.microsoft.com...
> I'm responsible for a pair of IIS 6 webservers that run our production
> ASP.Net application. We push code on a monthly basis, and about two ws
> after our October code push, we started to run into occasional webserver
> errors.
> The application pool is configured to run as a domain user which has
> permissions to the web code. When we have issues, the webserver will
> return the ASP.Net "Server Application Unavailable" page, and our app pool
> memory usage is really high (anywhere from 2 to 4 times our normal usage).
> Non asp.net content is served up just fine. If I recycle the application
> pool or do an iisreset, memory usage drops back to normal and the site
> starts working again.
> Nothing is logged to the eventlog or http error log, and the only errors
> we see from our application has been the occasional "Exception of type
> System.OutOfMemoryException was thrown." The same page doesn't error
> every time, and the function I've seen throw exceptions is incredibly
> generic and used heavily throughout our entire application (or so our devs
> have told me). Our application doesn't even log an error every time this
> has happened.
> How should I go about troubleshooting this? Our devs are saying it's a
> server issue, but I'm leaning towards a code issue. These servers are
> running Windows 2003 Server standard, fully updated as of this last
> wend.
We're using the session state db.
"Eliyahu Goldin" <REMOVEALLCAPITALSeEgGoldDinN@.mMvVpPsS.org> wrote in
message news:e0N$Dc2JIHA.2064@.TK2MSFTNGP06.phx.gbl...
> What session management are you using. InProc sessions are major memory
> consumers.
> --
> Eliyahu Goldin,
> Software Developer
> Microsoft MVP [ASP.NET]
> http://msmvps.com/blogs/egoldin
> http://usableasp.net
>
> "Daniel Peterson" <pythas@.hotmail.com> wrote in message
> news:CB009C9F-B38F-4937-8AF9-256502E90CDE@.microsoft.com...
>
So what's the best route to troubleshoot, just bust out debugdiag and go
from there?
If it's option #2 as you mention below, would cycling the application pool
more frequently solve that? I think we're using the 1740 minute default
right now.
"Kevin Spencer" <unclechutney@.nothinks.com> wrote in message
news:#YiaFK4JIHA.5360@.TK2MSFTNGP03.phx.gbl...
> Hi Daniel,
> It's a code issue. It will be one of 2 possible reasons: 1. Memory Leak.
> Yes, .Net applications can leak memory, if they use unmanaged resources
> that are not disposed properly. 2. Memory Fragmentation. If the same block
> of memory is re-used as, for example, an application-level collection that
> has members added and removed over a long period of time, that memory can
> become fragmented, necessitating the allocation of more memory to create
> contiguous blocks when they are needed.
> --
> HTH,
> Kevin Spencer
> Chicken Salad Surgeon
> Microsoft MVP
> "Daniel Peterson" <pythas@.hotmail.com> wrote in message
> news:CB009C9F-B38F-4937-8AF9-256502E90CDE@.microsoft.com...
>
Hi Daniel,
You could set up some performance counters, but the first thing I think I
would do is look for evidence of the 2 conditions I specified in the code.
It might very well show up rather easily if you know what you're looking
for.
HTH,
Kevin Spencer
Chicken Salad Surgeon
Microsoft MVP
"Daniel Peterson" <pythas@.hotmail.com> wrote in message
news:0323BA74-E5C3-4FEA-9C05-38B520DA602D@.microsoft.com...
> So what's the best route to troubleshoot, just bust out debugdiag and go
> from there?
> If it's option #2 as you mention below, would cycling the application pool
> more frequently solve that? I think we're using the 1740 minute default
> right now.
> "Kevin Spencer" <unclechutney@.nothinks.com> wrote in message
> news:#YiaFK4JIHA.5360@.TK2MSFTNGP03.phx.gbl...
I agree with Kevin, it's a code issue. Too many coders think they don't have
to worry about memory management anymore since "DotNet does it for me!".
Personally, I would start by looking at the Session and/or Application
object. It's likely that some coder decided it would be convenient to stuff
a huge collection of search results (or some such thing) into the session
object and now it's being held there until the session expires.

> You could set up some performance counters, but the first thing I think I
> would do is look for evidence of the 2 conditions I specified in the code.
> It might very well show up rather easily if you know what you're looking
> for.
Another culprit could be IDisposable objects not being disposed. You
doing anything with the graphics object maybe?
Dave Bush
http://blog.dmbcllc.com
--Original Message--
From: Scott Roberts [mailto:sroberts@.no.spam.here-webworks-software.com]
Posted At: Friday, November 16, 2007 9:34 AM
Posted To: microsoft.public.dotnet.framework.aspnet
Conversation: Troubleshooting ASP.Net memory usage issues?
Subject: Re: Troubleshooting ASP.Net memory usage issues?
I agree with Kevin, it's a code issue. Too many coders think they don't
have
to worry about memory management anymore since "DotNet does it for me!".
Personally, I would start by looking at the Session and/or Application
object. It's likely that some coder decided it would be convenient to
stuff
a huge collection of search results (or some such thing) into the
session
object and now it's being held there until the session expires.

> You could set up some performance counters, but the first thing I
think I
> would do is look for evidence of the 2 conditions I specified in the
code.
> It might very well show up rather easily if you know what you're
looking
> for.
Thanks for your help everyone. We found a 3rd party component with
debugdiag that appears to be leaking memory, although our devs seemed to
dismiss that as the issue, and they're still pointing to the servers. If
they don't want to look, I'm not sure how much more I can do.
They think that
http://support.microsoft.com/defaul...268&Product=asp
this is what's going on, although we're not seeing any of those errors or
eventlog messages.
Ugh.
"Scott Roberts" <sroberts@.no.spam.here-webworks-software.com> wrote in
message news:etWOA3FKIHA.5208@.TK2MSFTNGP04.phx.gbl...
> I agree with Kevin, it's a code issue. Too many coders think they don't
> have to worry about memory management anymore since "DotNet does it for
> me!".
> Personally, I would start by looking at the Session and/or Application
> object. It's likely that some coder decided it would be convenient to
> stuff a huge collection of search results (or some such thing) into the
> session object and now it's being held there until the session expires.
>
>
> Thanks for your help everyone. We found a 3rd party component with
> debugdiag that appears to be leaking memory, although our devs seemed to
> dismiss that as the issue, and they're still pointing to the servers. If
> they don't want to look, I'm not sure how much more I can do.
In your original post you mentioned that the problem started after a new
version was published, right? Was this 3rd party component used in the old
version of the site? Was it upgraded for the new version? Does the new
version use some features of this component that were previously unused?

> They think that
> http://support.microsoft.com/defaul...268&Product=asp
> this is what's going on, although we're not seeing any of those errors or
> eventlog messages.
I don't see anything pertaining to large memory usage in that KB. Is it a
memory problem or a thread pool problem?