Bug 138851 - Add Heap verification infrastructure
Summary: Add Heap verification infrastructure
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Mark Lam
URL:
Keywords:
Depends on: 138962
Blocks:
  Show dependency treegraph
 
Reported: 2014-11-18 16:02 PST by Mark Lam
Modified: 2014-11-21 05:39 PST (History)
5 users (show)

See Also:


Attachments
the patch (25.41 KB, patch)
2014-11-18 16:10 PST, Mark Lam
no flags Details | Formatted Diff | Diff
revised patch after Mark H's feedback (25.30 KB, patch)
2014-11-18 17:21 PST, Mark Lam
no flags Details | Formatted Diff | Diff
take 3: with some needed #include of inline files. (25.39 KB, patch)
2014-11-18 17:53 PST, Mark Lam
no flags Details | Formatted Diff | Diff
take 4: plus missing build file changes for other platforms. (27.82 KB, patch)
2014-11-18 18:17 PST, Mark Lam
no flags Details | Formatted Diff | Diff
take 5: speculative fix for VC++'s short coming. Uploading to test on Win EWS. (27.80 KB, patch)
2014-11-18 18:37 PST, Mark Lam
no flags Details | Formatted Diff | Diff
take 6: suppress compiler warnings because EFL compiler is not smart enough to detect a dead branch. (27.97 KB, patch)
2014-11-19 08:41 PST, Mark Lam
ggaren: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Mark Lam 2014-11-18 16:02:25 PST
This is debugging code I wrote for a prior debugging session.  Just cleaning it up for landing so that we can reuse / build on it in the future as needed.

The code is always build in but disabled by default.  When disabled, the cost is minimal:
1. Heap has a m_verifier field.
2. GC does a few "if (m_verifier)" checks that should fail.
3. HeapVerifier takes up code space though not used.

When enabled:
1. The HeapVerifier will keep N number of GC pass data.
   Each GC pass will contain a "before marking" and "after marking" live object list.
   The GC passes is a circular buffer.  Only data for the last N GC passes will be retained.
2. During GC, the current GC pass' live objects lists will be populated before and after marking.
3. The current GC pass' live object lists will be validated before GC, after marking, and after GC.

Currently, the only validation being done is to verify that object butterflies are allocated from valid blocks in the Storage (aka Copied) space.
Comment 1 Mark Lam 2014-11-18 16:10:44 PST
Created attachment 241826 [details]
the patch
Comment 2 Mark Hahnenberg 2014-11-18 16:30:41 PST
Comment on attachment 241826 [details]
the patch

View in context: https://bugs.webkit.org/attachment.cgi?id=241826&action=review

Did this find a bug?

> Source/JavaScriptCore/heap/Heap.cpp:1015
> +    if (m_verifier) {
> +        m_verifier->initializeGCPass();
> +        m_verifier->gatherLiveObjects(HeapVerifier::Phase::BeforeMarking);
> +    }

Sort of a nit, but this is done after the start of GC, but the AfterGC portion of the verification occurs after the final call to currentTimeMS. Might be more consistent to do both during the "official" collection.

> Source/JavaScriptCore/heap/HeapVerifier.cpp:197
> +    // dataLogF("Verifying @ phase %s, object list '%s' (size %zu)\n", phaseName(phase), list.name, liveObjects.size());

Remove or uncomment.

> Source/JavaScriptCore/runtime/Options.h:291
> +    v(unsigned, numberOfGCPassesToRecordForVerification, 3) \

"Pass" is sort of an odd term for this. I'm not sure I know the difference between a "Pass" and a "Phase". Not sure what else to call it though :-/ Maybe..."Cycle"?
Comment 3 Mark Lam 2014-11-18 17:20:23 PST
(In reply to comment #2)
> Comment on attachment 241826 [details]
> the patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=241826&action=review
> 
> Did this find a bug?

Yes.  It found a tricky write barrier bug.

> > Source/JavaScriptCore/heap/Heap.cpp:1015
> > +    if (m_verifier) {
> > +        m_verifier->initializeGCPass();
> > +        m_verifier->gatherLiveObjects(HeapVerifier::Phase::BeforeMarking);
> > +    }
> 
> Sort of a nit, but this is done after the start of GC, but the AfterGC
> portion of the verification occurs after the final call to currentTimeMS.
> Might be more consistent to do both during the "official" collection.

I've moved the BeforeGC verification above this into this block and similarly moved the AfterGC verification before we sample the GC end time.

> > Source/JavaScriptCore/heap/HeapVerifier.cpp:197
> > +    // dataLogF("Verifying @ phase %s, object list '%s' (size %zu)\n", phaseName(phase), list.name, liveObjects.size());
> 
> Remove or uncomment.

Thought this might be useful but may be verbose if printed all time.  Hence, I left it there commented out for convenience, but have removed it now since it's not hard to add it back when I want it.

> > Source/JavaScriptCore/runtime/Options.h:291
> > +    v(unsigned, numberOfGCPassesToRecordForVerification, 3) \
> 
> "Pass" is sort of an odd term for this. I'm not sure I know the difference
> between a "Pass" and a "Phase". Not sure what else to call it though :-/
> Maybe..."Cycle"?

I think GCPass is easier to read then GCCycle (both mean the same thing), but we already use the term cycle elsewhere in the GC code.  I renamed Pass to Cycle.
Comment 4 Mark Lam 2014-11-18 17:21:27 PST
Created attachment 241837 [details]
revised patch after Mark H's feedback
Comment 5 Mark Lam 2014-11-18 17:53:15 PST
Created attachment 241841 [details]
take 3: with some needed #include of inline files.
Comment 6 Mark Lam 2014-11-18 18:17:14 PST
Created attachment 241842 [details]
take 4: plus missing build file changes for other platforms.
Comment 7 Mark Lam 2014-11-18 18:35:07 PST
The Win EWS is barfing because:

     1>..\heap\HeapVerifier.cpp(227): fatal error C1001: An internal error has occurred in the compiler.
         (compiler file 'msc1.cpp', line 1325)
          To work around this problem, try simplifying or changing the program near the locations listed above.
         Please choose the Technical Support command on the Visual C++ 
          Help menu, or open the Technical Support help file for more information

In other words, VC++ fail.  Will upload some speculative patches to see if I can work around VC++'s short coming.
Comment 8 Mark Lam 2014-11-18 18:37:10 PST
Created attachment 241843 [details]
take 5: speculative fix for VC++'s short coming. Uploading to test on Win EWS.
Comment 9 Mark Lam 2014-11-19 08:41:03 PST
Created attachment 241861 [details]
take 6: suppress compiler warnings because EFL compiler is not smart enough to detect a dead branch.
Comment 10 Geoffrey Garen 2014-11-20 15:16:14 PST
Comment on attachment 241861 [details]
take 6: suppress compiler warnings because EFL compiler is not smart enough to detect a dead branch.

r=me if it you get it to build -- looks like the Windows build is still broken.
Comment 11 Mark Lam 2014-11-20 15:19:08 PST
(In reply to comment #10)
> r=me if it you get it to build -- looks like the Windows build is still
> broken.

The Windows EWS was unable to build even without my patch. The 5th version of my patch shows that my changes do build on Windows.  Will land.
Comment 12 Mark Lam 2014-11-20 15:29:26 PST
Thanks for the review.  Landed in r176424: <http://trac.webkit.org/r176424>.