Fast turnaround for leak bugs
I turned on trace-refcnt a few days ago in order to find leak bugs. I found three yesterday and three today.
Four of the six bugs already have patches :)
I turned on trace-refcnt a few days ago in order to find leak bugs. I found three yesterday and three today.
Four of the six bugs already have patches :)
September 23rd, 2007 at 12:44 pm
My experiences in helping people out at the MozillaZine forums have led me to conclude that most people who are having a problem with Firefox are not experiencing a bug in Firefox. Most often, I cannot reproduce the problems they are having, and their problems can be solved by following the advice in the MozillaZine Knowledge Base. One clear example of this effect is the number of people complaining that Firefox uses 100% of their CPU, as in the thread . Although there are some bugs in Firefox that cause it to use lots of CPU, very few people who experience Firefox using lots of CPU are experiencing a bug in Firefox. Generally, they are experiencing a problem with an extension, a video driver, or are just on a page that has a Flash animation.
Problems with Firefox using lots of memory are no exception. Many people complain about Firefox using lots of memory, but for the past year I have not been able to reproduce even one problem they are describing. Creating a new profile generally fixes the problem they were having, if they were having any problem to begin with. Now I know Firefox does have some memory leaks. I have reported a few myself, and I notice that after using Firefox for about a week that its memory use will not drop below 200 MB. However, I notice lots of people complaining that Firefox uses hundreds of megabytes within a day or two, and they’re always having to restart Firefox. I can’t believe that any significant fraction of these people are experiencing memory leaks in Firefox itself. The main offender seems to be bad memory leaks in extensions.
The end result of saying you’re working on fixing memory leak bugs in Firefox is that when these people try out the fixed version they don’t see a drop in memory use. Of course not, because they were not experiencing a bug in Firefox in the first place. They go away saying “Hopefully they fix it…but they keep saying that every release.” (quote taken from the Digg article). Further, you just reinforce the idea that Firefox has memory leaks, which many people take to mean a horrible, obvious memory problem that causes Firefox to use much more memory than other browsers. As far as I can tell, Firefox 2 already uses less memory than other browsers as I describe in the thread .
On the one hand, it’s good to work on memory leaks in Firefox. On the other hand, saying you’re working on memory leaks to Firefox users in general probably just sets users up to be disappointed. It reinforces the connection between Firefox and memory leaks, leading people to believe that Firefox has a serious problem dealing with memory.
Here are some suggestions I have:
Instead of communicating the memory leak information with Firefox users in general, target the hardcore testers by discussing in in the Firefox Bugs or Firefox Builds forum at MozillaZine.
In addition to working on memory leaks, put more effort into hang and crash bugs and other stability problems. When I try to use Firefox for an extended period of time (over a week), the problem that trips me up is never a memory problem, but another kind of stability problem. Even if I had only 512 MB of RAM, the minimum recommended on new Windows and Mac systems for years, it would be weeks before I would have any problem with memory — the hangs and crashes seem to be much worse of a problem to me.
Work together with extension developers to fix memory leaks in their extensions. This also goes for other stability problems in extensions, such as CPU problems and hangs.
Send users to the MozillaZine Knowledge Base or the new official Mozilla support site when they experience problems.
On the other hand, if you’re reading this and are infuriated that we still don’t see the horrible, obvious memory problem you’re having, post the details of your problem in the Firefox Bugs forum at MozillaZine. If you can show me how to reproduce a memory problem, I’ll be sure a bug report is filed in Bugzilla.
September 23rd, 2007 at 1:45 pm
Do you know if the “XPCOM_MEM_LEAK_LOG=2” setting is compatible with the flags passed in for use by leaks-gauge.pl? Those flags are: “NSPR_LOG_MODULES=DOMLeak:5,DocumentLeak:5,nsDocShellLeak:5”.
I will have to see what extra information the XPCOM_MEM_LEAK_LOG variable gives and if the output format would be hard for leaks-gauge to digest.
I wish I knew a way to search for environment settings, such as XPCOM_MEM_LEAK_LOG. Perhaps there is a way. If there is one way that js code reads the environment, and one way that C++ code reads the environment, it would be possible to search for those. Admittedly, I have not looked yet, but the code is not usually that regular. The only rule I see in Moz code is that there are very few rules and any rules have lots of exceptions.
September 25th, 2007 at 9:41 am
How Can I Help Because I Have A Lot Of Extensions That I Use. Close To 200 Theme’s & Extensions
September 26th, 2007 at 8:16 pm
I just saw the Slashdot article. Looking through the comments, it appears that most readers still think Firefox has horrible problems with managing memory. Reassuring Firefox users that memory leaks are being fixed is having the opposite of the desired effect — it convinces them the problem is terrible and is not being fixed.
I’m convinced that even if all memory leak bugs in Firefox are fixed, users will still complain just as fervently as ever. Of course, you still want to go ahead and fix the bad leaks, just don’t overemphasize memory optimization to the exclusion of even worse bugs. Talking publicly about memory leak fixes seems like it has a negative impact on user satisfaction with Firefox. It just reinforces the idea that Firefox does a poor job of dealing with memory.
What do others think?
September 27th, 2007 at 7:17 pm
Steve, you make an interesting argument, but I think there are some compelling reasons to talk about memory leaks despite the confusion it might cause.
1. Getting more of the Mozilla community up to speed on how we think about leaks is the only way we can really solve the problem of leaky extensions. David Baron and Steve England and I can’t test a large number of extensions thoroughly, but nightly users and addons.mozilla.org testers can, if they know what to look for.
2. Some of these changes do help with leaks caused by extensions, if indirectly. Having a baseline of RLk=0 on Mac and Linux will make it easier for extension authors and AMO testers to understand the output from using XPCOM_MEM_LEAK_LOG. Having all of our leak-detection tools work on Mac and Windows will allow more extension authors to test for leaks thoroughly. Some cycle collector changes might make extensions that leak in Firefox 2 not leak in Firefox 3, even if it was considered “the extension’s fault” that it leaked in Firefox 2.
3. Having fewer leaks (and lower memory usage) in Firefox itself will make it easier to make the argument that most memory leak problems users see while using Firefox 3 are due to extensions.
4. These nuances may be lost on some Slashdot users, but I don’t think that’s enough of a reason to avoid discussing these things in public and even on Slashdot. The smarter Slashdot users will get it. If you worry too much about part of your sentences being taken out of context and used to attack you later, you end not saying anything meaningful about complex issues, making it hard for smart people to support you enthusiastically. Software doesn’t have to be like US politics.
October 1st, 2007 at 7:14 pm
Jesse I have got a rig going to test memory leaks and I’ve come up with my first result:
Leaked document at address 2459c90.
… with URI “chrome://browser/content/browser.xul”.
Summary:
Leaked 0 out of 48 DOM Windows
Leaked 1 out of 165 documents
Leaked 0 out of 21 docshells
What do I do now?
October 1st, 2007 at 7:24 pm
pd, there are a lot of things that could cause the browser’s chrome document to leak. To know what to fix, we’d need steps to reproduce the leak, but I’m guessing that session was too long for you to remember what you did. Shorter sessions help with that kind of thing. Using trace-refcnt might help too, but you’d have to ask dbaron.
October 3rd, 2007 at 5:43 am
Hi Jesse
Submitted my first mlk bug last night with help from people in #developers on irc.mozilla.org
Dave Townsend pointed out this:
Environment variables affecting crash reporting
Which helped me get over a stumbling block where Breakpad kicked in every time I tried to run the latest nightly with the NSPR environment variables. I added this:
SET MOZ_CRASHREPORTER_DISABLE
to the start of my batch file to do the trick.
Also pointing out was this Debugging memory leaks document. However would I be right in assuming the tips in this document require C++ programming skills to implement?
Please vote for my bug :)
October 3rd, 2007 at 8:16 pm
Odd, what environment variable causes breakpad to appear? Does it incorrectly think Firefox has crashed?
October 4th, 2007 at 2:38 am
Jesse,
I have a question whether some stuff is regarded as a memory like by the Firefox developers:
If you open a tab, browser around a bit and then close a tab, the Firefox process is consuming more memory then before the tab was opened (even with browser.sessionhistory.max_total_viewers set to 0).
Does this mean Firefox is leaking memory? Or are the other reasons memory consumption could have increased (loaded plugins)?
October 4th, 2007 at 3:36 pm
Arjen, there are many reasons that might happen: malloc fragmentation, caching (reasonable or unreasonable), or a leak.
First of all, you’re using a recent trunk version of Firefox, right? Because looking for leaks in Firefox 2 is more or less pointless at this time :)
For large increases in memory usage, an easy test is to do is to do the same thing (open a tab, load the page in question, close the tab) five or ten times, watching memory usage. Does memory use keep going up, or does it level off? If it levels off, it’s not a leak, and is probably caching or malloc fragmentation.
If you see that memory usage keeps going up, there’s a good chance you’ve found a leak. You could file a bug report at this point, noting that memory use keeps going up, but the bug is more likely to be fixed quickly if you include information from one of our leak tools showing that it’s definitely a leak.
If leak-gauge.pl or trace-refcnt or trace-malloc says what objects leak, you have a great bug report and it’s very likely to be fixed before we ship Firefox 3. leak-gauge.pl is pretty easy to run but only finds certain large leaks; the others are progressively harder to run and find progressively “smaller” leaks.
If none of those tools shows a leak, it could still be a leak-until-shutdown. These bugs are harder to diagnose; I think the preferred tool is diffbloatdump.pl (which requires a trace-malloc build).
Having simple steps-to-reproduce is important, too. Having a simple testcase is nice but isn’t as important for leak bugs as it is for most other types of bugs.
October 8th, 2007 at 1:32 am
Hi,
Congratulation on starting this memory leak roundup.
How about running static code analyzers as well on Mozilla code? This may sound a bit old fashioned however testing for bugs is rather limited by test-cases, scenarios and tools available. Running static code analyzer can probably be worth trying along with these.