Bug 108410 - Vector should consult allocator about ideal size when choosing capacity.
Summary: Vector should consult allocator about ideal size when choosing capacity.
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Template Framework (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Andreas Kling
URL:
Keywords: InRadar, Performance
Depends on:
Blocks:
 
Reported: 2013-01-30 17:51 PST by Andreas Kling
Modified: 2013-02-04 09:13 PST (History)
13 users (show)

See Also:


Attachments
Patch (5.66 KB, patch)
2013-01-30 17:57 PST, Andreas Kling
no flags Details | Formatted Diff | Diff
Patch (6.46 KB, patch)
2013-01-30 18:15 PST, Andreas Kling
no flags Details | Formatted Diff | Diff
Patch (6.84 KB, patch)
2013-01-30 18:34 PST, Andreas Kling
benjamin: review+
buildbot: commit-queue-
Details | Formatted Diff | Diff
Patch (8.42 KB, patch)
2013-02-02 16:15 PST, Andreas Kling
benjamin: review+
buildbot: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Andreas Kling 2013-01-30 17:51:05 PST
Vector should consult allocator about ideal size when choosing capacity.
Comment 1 Andreas Kling 2013-01-30 17:57:10 PST
Created attachment 185640 [details]
Patch
Comment 2 Benjamin Poulain 2013-01-30 18:09:19 PST
Comment on attachment 185640 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=185640&action=review

> Source/WTF/wtf/Vector.h:255
> +            newCapacity = fastMallocGoodSize(newCapacity * sizeof(T)) / sizeof(T);

Why not also tryAllocateBuffer() and reallocateBuffer()?
Comment 3 Andreas Kling 2013-01-30 18:11:21 PST
(In reply to comment #2)
> (From update of attachment 185640 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=185640&action=review
> 
> > Source/WTF/wtf/Vector.h:255
> > +            newCapacity = fastMallocGoodSize(newCapacity * sizeof(T)) / sizeof(T);
> 
> Why not also tryAllocateBuffer() and reallocateBuffer()?

Because blind spot. :(
Comment 4 Andreas Kling 2013-01-30 18:15:06 PST
Created attachment 185645 [details]
Patch

Kinda curious how EWS likes this.
Comment 5 Benjamin Poulain 2013-01-30 18:23:08 PST
Comment on attachment 185645 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=185645&action=review

I love this patch.

Overflow?

> Source/JavaScriptCore/ChangeLog:8
> +        Remove assertion about Vector capacity that won't hold anymore since.

"that won't hold anymore since."?

> Source/WTF/wtf/Deque.h:387
> -        size_t newCapacity = std::max(static_cast<size_t>(16), oldCapacity + oldCapacity / 4 + 1);
>          T* oldBuffer = m_buffer.buffer();
> -        m_buffer.allocateBuffer(newCapacity);
> +        m_buffer.allocateBuffer(std::max(static_cast<size_t>(16), oldCapacity + oldCapacity / 4 + 1));

I liked the variable newCapacity :)
Comment 6 Andreas Kling 2013-01-30 18:34:53 PST
Created attachment 185652 [details]
Patch

So much thrashing. I blame the coffee.

@Benjamin: I removed the 'newCapacity' local in Deque since I have to read back m_buffer.capacity() for the correct value.
Comment 7 Benjamin Poulain 2013-01-30 18:41:49 PST
Comment on attachment 185652 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=185652&action=review

> Source/WTF/wtf/Vector.h:259
> +            newCapacity = fastMallocGoodSize(newCapacity * sizeof(T)) / sizeof(T);
> +            m_capacity = newCapacity;
>              m_buffer = static_cast<T*>(fastMalloc(newCapacity * sizeof(T)));

Okay, now, what about we can remove the /sizeof(T)? :-D

size_t sizeToAllocate = fastMallocGoodSize(newCapacity * sizeof(T));
m_capacity = sizeToAllocate / sizeof(T)
m_buffer = mallo(sizeToAllocate).

> Source/WTF/wtf/Vector.h:271
> +            newCapacity = fastMallocGoodSize(newCapacity * sizeof(T)) / sizeof(T);
>              T* newBuffer;
>              if (tryFastMalloc(newCapacity * sizeof(T)).getValue(newBuffer)) {
>                  m_capacity = newCapacity;

ditto

> Source/WTF/wtf/Vector.h:290
>              ASSERT(shouldReallocateBuffer(newCapacity));
> -            m_capacity = newCapacity;
>              if (newCapacity > std::numeric_limits<size_t>::max() / sizeof(T))
>                  CRASH();
> +            newCapacity = fastMallocGoodSize(newCapacity * sizeof(T)) / sizeof(T);
> +            m_capacity = newCapacity;
>              m_buffer = static_cast<T*>(fastRealloc(m_buffer, newCapacity * sizeof(T)));

ditto.
Comment 8 Andreas Kling 2013-01-30 18:46:12 PST
(In reply to comment #7)
> (From update of attachment 185652 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=185652&action=review
> 
> > Source/WTF/wtf/Vector.h:259
> > +            newCapacity = fastMallocGoodSize(newCapacity * sizeof(T)) / sizeof(T);
> > +            m_capacity = newCapacity;
> >              m_buffer = static_cast<T*>(fastMalloc(newCapacity * sizeof(T)));
> 
> Okay, now, what about we can remove the /sizeof(T)? :-D
> 
> size_t sizeToAllocate = fastMallocGoodSize(newCapacity * sizeof(T));
> m_capacity = sizeToAllocate / sizeof(T)
> m_buffer = mallo(sizeToAllocate).
> 
> > Source/WTF/wtf/Vector.h:271
> > +            newCapacity = fastMallocGoodSize(newCapacity * sizeof(T)) / sizeof(T);
> >              T* newBuffer;
> >              if (tryFastMalloc(newCapacity * sizeof(T)).getValue(newBuffer)) {
> >                  m_capacity = newCapacity;
> 
> ditto
> 
> > Source/WTF/wtf/Vector.h:290
> >              ASSERT(shouldReallocateBuffer(newCapacity));
> > -            m_capacity = newCapacity;
> >              if (newCapacity > std::numeric_limits<size_t>::max() / sizeof(T))
> >                  CRASH();
> > +            newCapacity = fastMallocGoodSize(newCapacity * sizeof(T)) / sizeof(T);
> > +            m_capacity = newCapacity;
> >              m_buffer = static_cast<T*>(fastRealloc(m_buffer, newCapacity * sizeof(T)));
> 
> ditto.

Sure, when I'm awake again. No need to reupload for that :)
Comment 9 Build Bot 2013-01-31 01:49:51 PST
Comment on attachment 185652 [details]
Patch

Attachment 185652 [details] did not pass win-ews (win):
Output: http://queues.webkit.org/results/16271120
Comment 10 Radar WebKit Bug Importer 2013-01-31 05:17:30 PST
<rdar://problem/13124002>
Comment 11 Andreas Kling 2013-01-31 05:36:11 PST
Committed r141407: <http://trac.webkit.org/changeset/141407>
Comment 12 Andreas Kling 2013-01-31 05:40:48 PST
(In reply to comment #9)
> (From update of attachment 185652 [details])
> Attachment 185652 [details] did not pass win-ews (win):
> Output: http://queues.webkit.org/results/16271120

Landed with Benjamin's comments addressed. This ews failure looks like a rebuild of QTMovieWin is needed.
Comment 13 Jessie Berlin 2013-01-31 16:40:15 PST
I had to roll this patch out in http://trac.webkit.org/changeset/141504 because it was causing crashes in release builds.
Comment 14 Andreas Kling 2013-02-02 16:15:37 PST
Created attachment 186239 [details]
Patch

Finally tracked down the crash, it was due to fastMallocGoodSize() getting called before the TCMalloc page heap was fully initialized.
Fixed this by calling TCMalloc_ThreadCache::InitModule() in fastMallocGoodSize() if needed.
Comment 15 Benjamin Poulain 2013-02-02 16:57:53 PST
Comment on attachment 186239 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=186239&action=review

> Source/WTF/wtf/FastMalloc.cpp:2574
> +    if (!phinited)
> +        TCMalloc_ThreadCache::InitModule();
> +    return AllocationSize(bytes);
> +}

Any reason we couldn't generate class_to_size and class_to_pages  at compile time?
Comment 16 Build Bot 2013-02-02 17:18:27 PST
Comment on attachment 186239 [details]
Patch

Attachment 186239 [details] did not pass win-ews (win):
Output: http://queues.webkit.org/results/16358080
Comment 17 Andreas Kling 2013-02-03 05:39:27 PST
Committed r141716: <http://trac.webkit.org/changeset/141716>
Comment 18 Andreas Kling 2013-02-03 05:44:53 PST
(In reply to comment #15)
> (From update of attachment 186239 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=186239&action=review
> 
> > Source/WTF/wtf/FastMalloc.cpp:2574
> > +    if (!phinited)
> > +        TCMalloc_ThreadCache::InitModule();
> > +    return AllocationSize(bytes);
> > +}
> 
> Any reason we couldn't generate class_to_size and class_to_pages  at compile time?

We might be able to do that. Let's take that in a separate patch though.
Comment 19 Darin Adler 2013-02-04 09:13:21 PST
I made an attempt at this for String back in 2008 when strings were resizable. I guess there’s a chance that StringBuilder might benefit from this, even today.