44a6527f33
ck_ring.h: make _ck_ring_enqueue_mp less failure happy
...
With preemption, it is possible for _ck_ring_enqueue_mp to have a
snapshot of p_head so stale with respect to the later snapshot of
c_head that a comparison modulo (small) ring size will erroneously
conclude that the ring is full.
Detect that case and retry rather than failing. We only retry when
the enqueuers have made global forward progress, so the first loop
is as lock-free as it ever was.
Bonus: the new condition should be marginally faster.
8 years ago
4f01918cbb
Update ck_ring.h
...
fix bug
8 years ago
f1302c4920
ck_ring: explicit casts to clarify the semantics of generic (pointer) dequeuing
9 years ago
27fb1bc00f
ck: Reimplement a few libc headers, to make CK build when compiling in the
...
FreeBSD kernel.
9 years ago
4677de8473
ck_ring: [whitespace] Add comment to PROTOTYPE family.
9 years ago
fee4e7187b
ck_ring: Clean-up internal implementations.
...
Break out internal implementations to _mp and _sc variants from which
public interface is built on. Do not rely on macro. Adopt CK_CC_RESTRICT
instead of using restrict directly.
9 years ago
cc600e7d4f
ck_ring: Add MPMC and MPSC variants.
9 years ago
e9d96eb9f6
ck_ring: make a few arguments const
10 years ago
554e2f0874
whitespace: Strictly conform to C namespacing rules.
10 years ago
0dfd145aa6
whitespace: Update Copyright messages.
10 years ago
d6ba2a6273
whitespace: Sweeping whitespace changes.
...
Removing trailing whitespaces and newlines.
10 years ago
cefd6bc9ec
ck_ring: Remove stale comment from pre-memcpy implementation.
11 years ago
396ceb16bb
ck_ring: Fix typo in type-specialized ck_ring_dequeue_tryspmc.
...
Thanks to Paul Khuong <pkhuong@appnexus> for patch.
11 years ago
c197b37df1
legal: Update Copyright statements.
11 years ago
057f719796
ck_ring: Add type-specialized ring.
11 years ago
90c9950356
ck_ring: Type-agnostic backend implementation.
...
This will serve to drive both pointer-based and type-specialized ring
implementations. Some full fences have also been removed as they are
unnecessary with the introduction of X_Y fence interface.
11 years ago
b6a1914085
ck_ring: Prefer treatment of ck_ring_buffer_t as an opaque type.
11 years ago
b6f085a62e
ck_ring: Minor style change.
11 years ago
3edb523da5
ck_ring: Move the ring buffer outside of the ck_ring_t
...
Remove the ring buffer from the struct ck_ring, it is now required to
explicitely pass the ring buffer for enqueue/dequeue. That can be useful for
systems with multiple address spaces.
11 years ago
5d8a273dbe
whitespace: Bulk whitespace changes.
11 years ago
1201e09577
ck_ring: Remove stale comment.
12 years ago
878b8e6321
ck_ring: Correct CK_RING_SPMC_ENQUEUE_SIZE typo.
12 years ago
67381af3c5
ck_ring: Expose *_size variants to type-specialized ck_ring.
...
Added:
- CK_RING_ENQUEUE_SPSC_SIZE
- CK_RING_ENQUEUE_SPMC_SIZE
12 years ago
45f648bd33
ck_ring: Add *_size_* ck_ring enqueue operations.
...
These variants of ck_ring_enqueue_* return the snapshot of queue
length with respect to the linearization point. This can be used to
extract ring size without incurring additional cacheline invalidation
overhead from the writer.
12 years ago
0598307186
whitespace: Various style changes.
12 years ago
33a9222923
legal: Update Copyright statements.
12 years ago
c4149c72b4
ck_ring: Use regular assignment in type specialized ck_ring initialization.
12 years ago
89cb712ffa
ck_ring: Memoize mask across all operations.
12 years ago
8519aaf771
ck_ring: Memoize mask for SPMC operations.
12 years ago
b54ae5c4ac
ck_ring: Add a work-around for compiler bug(s) in ck_ring_dequeue_spmc.
...
Both LLVM-backed compilers and GCC incorrectly treat
a barrier-sandwiched load as a loop invariant in dequeue_spmc.
Forcing volatile atomic load semantics generates the right
thing.
Thanks to Devon O'Dell and Abel Mathew for help in catching
this issue.
12 years ago
849f938973
ck_ring: Add parameterized version of ck_ring_trydequeue.
12 years ago
7a99585f9e
ck_ring: No need for volatile load on dequeue_spmc.
12 years ago
01dc51db65
ck_ring: Add trydequeue variant.
12 years ago
055a7fd5db
ck_ring: Add wrapper functions for parameterized SPMC ring.
12 years ago
68b33e56d8
ck_ring: Add parameterized version of SPMC ring.
12 years ago
7f35cbee1b
ck_ring: Use SPSC enqueue as SPMC enqueue.
12 years ago
5d1d850f44
ck_ring: Adopt semantics identical to spmc counter semantics.
12 years ago
77a3bd3126
ck_ring: Move overflow check to producer side.
...
unsigned int-sized counters will be used on producer
and consumer side. This is necessary to allow a ring
to be used on multiple workloads.
12 years ago
d595bafea7
ck_ring: Serialize producer snapshot with respect to consumer.
12 years ago
d705e02b76
ck_ring: Replace MPMC variant with SPMC variant for starters.
...
Immediate use-case is SPMC. Next is MPSC, then I will generalize
to MPMC (unfortunately, I don't have an algorithm that avoids CAS2
yet for MPMC).
12 years ago
bb48f602ff
ck_ring: Add note regarding correctness so people don't both verifying it.
...
Original version is susceptible to several problems, it is whack.
12 years ago
68e6752e53
ck_ring: Due to popular request, add MPMC variants of enqueue/dequeue.
...
Fencing semantics are heavier-weight than they need to be.
12 years ago
a06f28b77c
ck_ring: Remove useless comment.
12 years ago
955047a7d1
ck_{ring,fifo}: Add store fences while we lack load_store fences.
12 years ago
b1b19c1f74
ck_ring: Memory fence is overly pedantic for parameterized enqueue.
12 years ago
80c56d115b
ck_ring: Do not use volatile load for parametric ck_ring_size.
12 years ago
ad5e143bd5
ck_ring: Various improvements to readability.
...
No regressions in unit tests, so I'm dropping volatile
semantics for unnecessary volatile load/store on producer/consumer
side.
12 years ago
94c26494a6
ck_ring: Size is no longer necessary.
...
Patch submitted by Andrew Schorr.
12 years ago
3cf265cba0
all: Strip trailing whitespaces.
13 years ago
42b32c6e6a
ck_ring: Apply ck_ring_size logic to ck_ring_enqueue functions.
...
This bug fix was contributed by Matt Johnson <johnso87@...>.
13 years ago
Paul Khuong
铁哥
Paul Khuong
Olivier Houchard
Samy Al Bahra
William Light
Samy Al Bahra
Samy Al Bahra