f8b41af77c
x86_64: Require 32-bit immediate values for expansion.
...
Pointed out by Paul Khuong, who ran into issues with immediates
while working on improvements to ck_bitmap.
11 years ago
cefd6bc9ec
ck_ring: Remove stale comment from pre-memcpy implementation.
11 years ago
396ceb16bb
ck_ring: Fix typo in type-specialized ck_ring_dequeue_tryspmc.
...
Thanks to Paul Khuong <pkhuong@appnexus> for patch.
11 years ago
6e0f79c70e
ck_tflock: Initial check-in of reference implementation.
11 years ago
671d067ea0
ck_bytelock: Fix casting error affecting 32-bit architectures.
11 years ago
81f267ce62
Merge branch 'master' of git.concurrencykit.org:ck
11 years ago
b97b0021f9
ck_barrier: Just pass-through tournament barrier with only one participating thread.
11 years ago
15ce0501d2
ck_sequence: Add branch hint for short-circuit.
11 years ago
cb8f5f9ebb
ck_hp_fifo: Add deinit routine to allow reclaiming stub node.
11 years ago
c75e470ed4
ck_fifo_spsc: De-initialize from head.
11 years ago
06feae82f3
ck_hp_fifo: No need for volatile atomic stores on initialization.
11 years ago
5ec8191ded
ck_fifo: Add ck_fifo_*_deinit routines.
...
These allow for reclamation of garbage node in an empty FIFO.
Based on patch submitted by Rajesh R <rr01803@gm....>.
11 years ago
15ac0bc691
ck_bitmap: Fix length calculation error in ck_bitmap_union (crasher).
11 years ago
08be876fa6
ck_bytelock: Correctly compute byte sequence length for readers.
...
Thanks to Albi Kavo <albi.kavo@gm...> for spotting this.
11 years ago
190b204059
ck_bitmap: Drop _mpmc suffix and add union operation.
...
The _mpmc suffix is not useful. I see little benefit to specializing
the bitmap to begin with.
11 years ago
76e43825a7
Merge remote-tracking branch 'origin/cognet'
11 years ago
b8c801136a
ck_bytelock: Fix deadlock for unslotted reader workloads.
...
There is an off-by-one for slot ID sizeof(bytelock->readers) + 1.
This patch fixes the handling of this slot ID. Based off a patch
submitted by Albi Kavo <albi.kavo@gma....>.
11 years ago
59cedf10c6
ck_rhs: Add a read mostly mode.
...
Add a read-mostly mode, in which entries won't share cache lines with
associated datas (probes, probe_bound, etc).
This makes write operations slower, but make get faster.
11 years ago
bf686c0a42
ck_rhs: Add.
...
Introduce ck_rhs, a variant of ck_hs that uses the robin hood algorithm for
insertion, and backward shift delete for deletion.
11 years ago
c197b37df1
legal: Update Copyright statements.
11 years ago
0c4ed1d14f
ck_pflock: Use store barrier on init.
11 years ago
0353cdcb23
ck_rwlock: Migrate to acquire / release and other barrier improvements.
11 years ago
38c40315a2
ck_brlock: Use store barrier on initialization.
11 years ago
ac8397d8b5
ck_rwcohort: Migrate to acquire / release.
11 years ago
5280253974
ck_pflock: Use precise barrier in read_unlock.
11 years ago
febdb8c164
ck_brlock: Use precise barrier for read_unlock.
11 years ago
0c205740aa
ck_pflock: Migrate to acquire / release.
11 years ago
ad5e00cfb9
ck_epoch: Read-side latency improvements on x86.
11 years ago
cad7b53273
ck_epoch: Migrate to acquire / release.
11 years ago
e8051b79c2
ck_elide: Add comment indicating purposeful omission of barriers.
11 years ago
16f7568e5c
ck_cohort: Migrate to acquire / release.
11 years ago
44689f7e69
ck_bytelock: Migrate to acquire / release.
11 years ago
f314b52035
ck_brlock: Migrate to acquire/release.
11 years ago
0bbb6f0eff
ck_pr: Fix typo in preprocessor usage.
11 years ago
e57bfd5865
spinlock: Migrate to acquire / release interface.
11 years ago
661f3948ed
ck_pr: Add acquire and release fences.
11 years ago
057f719796
ck_ring: Add type-specialized ring.
11 years ago
90c9950356
ck_ring: Type-agnostic backend implementation.
...
This will serve to drive both pointer-based and type-specialized ring
implementations. Some full fences have also been removed as they are
unnecessary with the introduction of X_Y fence interface.
11 years ago
4af7571990
ck_pr/arm: Remove extraneous newline.
11 years ago
259fb5174d
ck_pr: Use the "dmb st" instruction on arm when available.
11 years ago
b32e1b24df
ck_spinlock: Use more finely-grained barrier for CLH.
11 years ago
eb59f6ead5
ck_ht: Add CK_HT_WORKLOAD_DELETE flag.
...
This is a port of the hash set delete workload
hint.
11 years ago
b17706f967
hclh: Assert my copyright.
11 years ago
df571ad331
ck_spinlock: Migrate lock implementations to directory.
11 years ago
1ecb37b28f
Merge branch 'cognet' of git.concurrencykit.org:ck
11 years ago
b6a1914085
ck_ring: Prefer treatment of ck_ring_buffer_t as an opaque type.
11 years ago
b6f085a62e
ck_ring: Minor style change.
11 years ago
e6c0ef01a5
ck_spinlock: Revert previous change, do not splice too early.
11 years ago
e8b26ec212
ck_spinlock: Minor readability changes to ck_hclh.
11 years ago
c219593967
ck_spinlock: First round of improvements to ck_hclh.
...
* Drop unnecessary volatile stores.
* Use more expressive ck_pr_fence_X_Y semantics.
11 years ago
0e5ce1aae7
ck_spinlock: Simple whitespace changes (wrap long lines).
11 years ago
0f92cbab08
ck_spinlock: Commit the missing HCLH bits.
11 years ago
aa0c303554
ck_ht: Add ck_ht_gc functionality, similar to ck_hs_gc.
...
This allows for incremental reprobe to shorten over-all probe sequences
in delete heavy workloads.
11 years ago
3edb523da5
ck_ring: Move the ring buffer outside of the ck_ring_t
...
Remove the ring buffer from the struct ck_ring, it is now required to
explicitely pass the ring buffer for enqueue/dequeue. That can be useful for
systems with multiple address spaces.
11 years ago
b778a32b0e
Merge branch 'master' of git.concurrencykit.org:ck
11 years ago
0d6a0508cc
ck_hs: Allow for amortization and partial rebuild in ck_hs_gc.
11 years ago
f7eff4d61b
ck_hs: Add ck_hs_gc, mechanism for tombstone clean-up.
...
This is treated as an explicit garbage collection step
which will block writers.
11 years ago
5d4f9ae23d
ck_hs: Add ck_hs_rebuild operation.
...
This operation is short-hand notation for rebuilding
a hash table. This rebuild can occur in the presence
of concurrent readers and will require twice the amount
of memory of the existing hash table until completion.
11 years ago
2ad920cd85
ck_hs: Add improved support for delete-heavy workloads.
...
This borrows from a technique described by Purcell and Harris
in "Non-blocking hashtables with open addressing" technical report.
Essentially, every slot will have an associated local probe maxim,
including tombstones. Highly aggressive workloads may still require
occassional garbage collection.
11 years ago
30d956d682
ck_hs: Re-order includes in header file.
11 years ago
a3532d52ce
ck_hs: Add ck_hs_put_unique function for fast-path unique inserts.
...
This function allows for faster insertions into tombstone-heavy
probe sequences by short-circuiting on tombstones rather than
continuing to probe. The user must already guarantee that the
entry being inserted is unique. If a non-unique key is inserted
with this operation, undefined behavior will result.
11 years ago
0610a0d0a8
ck_bag: Drop support for this data structure.
...
Could not find suitable use-case and generally doesn't
appear interesting to academics in the existing
form. Maybe it will make a come-back in the future with
fewer memory and latency compromises.
11 years ago
3c089d55e5
arm: Add support for cas_2 and other double-word operations.
...
This adds support for CAS_64{_VALUE}, CAS_PTR_2{_VALUE},
LOAD_64, STORE_64 and other primitives built on universal
CAS primitive.
Patch submitted by Olivier Houchard <cognet@FreeBSD>.
11 years ago
b801ec4b98
ck_array: Remove variable name from ck_array_init declaration.
11 years ago
a7cb0f9bbb
ck_array: Add ck_array_initialized functionality.
...
The function returns true if the target is initialized, otherwise,
it returns false.
11 years ago
2db9de633f
ck_array: Simple eliminating SPMC dense array for fast iteration.
...
The array is optimized for SPMC and fast iteration (though MPMC
transformation is also possible). This is an extremely simple
implementation with support for atomic in-place modification
through put -> remove elimination.
11 years ago
5916485e9e
ck_pr/arm: Implement support for Thumb 2 and fix incorrect CAS behavior.
...
Besides implementing Thumb 2 supports, this fixes incorrect usage of
"cmp" where "cmpeq" was meant.
Patch submitted by Olivier Houchard <cognet@freebsd>.
11 years ago
e1c0f6854d
ck_pr/arm: Additional style clean-up.
11 years ago
1c97766039
ck_pr: First cut 32-bit ARM port.
...
Contributed by Olivier Houchard <cognet@freebsd>.
Signed-off-by: Samy Al Bahra <sbahra@repnop.org>
11 years ago
1cc189b08e
ck_bitmap: Remove unused argument for CK_BITMAP_BITS.
...
This is an API change, has been on the queue for a while.
Patch submitted by Shreyas Prasad <sprasad@appnexus>.
11 years ago
5d8a273dbe
whitespace: Bulk whitespace changes.
11 years ago
3e5dcfe6fd
ck_sequence: Add CK_SEQUENCE_READ operation.
11 years ago
2d4524dd3e
ck_spinlock: Retab (whitespace).
...
Accidentally had Haskell vim modeline set.
11 years ago
07aab518f6
ck_spinlock: Retab (whitespace).
11 years ago
ce142f17f8
ck_spinlock: Add ck_spinlock_mcs_init operation.
11 years ago
6bc9f7a485
ck_spinlock: Get rid of MCS context initializers.
...
They are unnecessary, legacy from older versions.
11 years ago
ec58f5dcf7
ck_spinlock: Add ck_spinlock_dec_init operation.
11 years ago
2f52ace54c
ck_spinlock: Fix strict aliasing issue in unlock (no-op).
...
For builds with -Werror. Not pointed out by Wez Furlong <wez@f....>.
12 years ago
29a84b47b3
ck_hs: Add ck_hs_move operation.
...
This operation moves ownership from one hash set object
to another and re-assigns callback functions to developer-specified
values. This allows for dynamic configuration of allocation
callbacks and is necessary for use-cases involving executable code
which may be unmapped underneath the hash set.
The developer is responsible for enforcing barriers and enforcing
the visibility of the new hash set.
12 years ago
5cf28b2152
ck_rwcohort: Missing fences on fast path.
...
When spinning on global counters, it cannot be assumed that is_locked
functions will guarantee atomic to load ordering, an explicit fence
is necessary. is_locked will only guarantee load ordering.
12 years ago
73a1e42baf
ck_rwcohort: Style fixes.
12 years ago
b36e35ec76
Merge branch 'master' of https://github.com/bscheinman/ck
12 years ago
ce2ef37ce3
ck_elide: Style fixes.
12 years ago
cc231e4ab0
ck_elide: No need to expose ck_elide_stat.
12 years ago
906eaf1881
ck_elide: Fold CK_ELIDE_ADAPTIVE_PROTOTYPE into PROTOTYPE.
12 years ago
b67e91c858
ck_rwlock: Fold CK_ELIDE_ADAPTIVE_PROTOTYPE into PROTOTYPE.
12 years ago
768b5b3597
ck_rwlock: Add adaptive elision and unit tests.
12 years ago
fe8deba87d
ck_elide: Adaptive elision semantics.
...
These come in the form of CK_ELIDE_ADAPTIVE_PROTOTYPE,
CK_ELIDE_LOCK_ADAPTIVE and CK_ELIDE_UNLOCK_ADAPTIVE.
Primarily pushing this for the few that are playing with
master.
This is inspired by Andi Kleen's work for adaptive behavior
in the Linux kernel's RTM locks implementation. There are
various differences in the state machine, however. Specifically,
the concept of a retry and a busy-wait has been unified due
to state machine simplification such that any exhausted busy-wait
cycle reverts to a forfeit (a busy-wait is a specialized retry).
Follow-up work will involve allowing for is_locked behavior
to yield what users expect, if called from with-in a transaction
through the wrapper. It is warned that this will come at a performance
penalty.
12 years ago
43f7a7692a
ck_cc: Expand LIKELY/UNLIKELY on unsupport compilers.
12 years ago
f214d9419e
ck_spinlock: Move elision prototype to end of file.
12 years ago
c76441fff3
ck_stack: Style changes.
12 years ago
fc0bfae1da
ck_elide: Commentary on ck_elide interface.
12 years ago
59a1b53e86
ck_spinlock: Migrate to ck_elide.h.
...
Pending unit test updates.
12 years ago
671f82d7a8
ck_rwlock: Migrate to ck_elide.h.
12 years ago
68a614328a
ck_elide: General lock elision framework.
...
This allows users to wrap any single-argument lock implementation
to support lock elision. Built-in lock types will make use of this
facility.
12 years ago
0517a51b06
ck_spinlock: Weaken volatile store in clh_lock.
12 years ago
64b1201dcc
ck_spinlock: Acquire semantics for old locked operations.
...
Even though the intended use-case is for testing, developers
may want correctness guarantees in other situations.
12 years ago
a51c8c7aa6
ck_spinlock: Add ck_spinlock_clh_locked.
12 years ago
46d61ccd46
ck_spinlock: Add ck_spinlock_ticket_locked.
12 years ago
f149dd09fe
ck_pr: Add ck_pr_fence_strict_atomic for x86-64.
12 years ago
55decd0b38
ck_spinlock: Add ck_spinlock_anderson_locked.
12 years ago
a03d58fff2
ck_rwlock: Use heavier-weight barrier on write path.
...
This is an example limitation of fence_X_Y variant. I am
considering extending this to include an acquire extension.
Use a memory fence to force total order in a manner that
will be clearer to other developers who read this.
This did not manifest as a problem on any target architectures
due to their handling of atomic operations (SPARC models it as
both a load and a store, while Power atomic_load ordering was
enforced through a full barrier).
12 years ago
a694e871ca
ck_rwlock: Relax write_trylock_rtm semantics.
...
trylock semantics are best effort to begin with, so
avoid fallback path all together.
12 years ago
3fbefc5042
ck_rwlock: Add read_trylock_rtm.
12 years ago
4d2ccfe497
ck_rwlock: Add basic RTM interface to rwlock.
...
It is possible this will be moved to a self-contained file.
For a majority of architectures, RTM is an unnecessary
implementation-specific optimization.
12 years ago
2ed965bfc2
ck_pr_rtm: Fix xtest output clobber identifier.
12 years ago
43a6628cf1
ck_pr: Initial support for TSX.
...
goto label support has been omitted for the moment.
A majority of initial targets are still pre-gcc-4.7
to support multiple branches.
12 years ago
04590839d9
ck_sequence: change retry to take unsigned int.
...
version identifier is unsigned int not uint32_t.
12 years ago
cb0a7c8ce6
Merge branch 'master' of https://github.com/sbahra/ck
12 years ago
a8a74b9f57
ck_spinlock: Remove erroneous mask usage for non-x86 targets.
12 years ago
045d0e1053
ck_spinlock: Apply modulo arithmetic to proportional back-off ticket lock.
12 years ago
cd53e928bf
ck_cc: Add branch execution hint support for GCC.
12 years ago
7da381bb1a
ck_hs: Prototype for new ck_hs_fas operation.
12 years ago
fc071e1ce2
Merge branch 'master' of https://github.com/bscheinman/ck
12 years ago
f9ea659bff
Merge branch 'master' of https://github.com/sbahra/ck
12 years ago
3ca7072c14
ck_pr: Implicit compiler barrier ck_pr_fence.
...
I accidentally removed ck_pr_fence implicit compiler
barrier semantics in re-structure of ck_pr_fence.
This does affect the correctness of any data structures
in ck_pr_fence or the correctness of consumers of ck_pr
operations where ck_pr serves as linearization points.
The reason it does not affect any CK data structures is
that explicit compiler barriers (whether they are store/load
operations or atomic ready-modify-write operations) always
serve as linearization points.
However, if consumers are doing tricky things like using
these barriers to serialize aliased locations for correctness,
then it is possible for compiler re-ordering to bite them in
the ass.
12 years ago
a5e8d6ad45
ck_spinlock: Only enable trylock for ck_spinlock_ticket on x86*.
...
trylock algorithm semantics are not cleanly defined outside of x86.
12 years ago
adbdfe6633
ck_pr: Get rid of ck_pr_fence_X_X functions.
...
These add unnecessary complexity to the ck_pr_fence interface.
Instead, it can be safely assumed that developers will use
ck_pr_fence_X to enforce X -> X ordering.
12 years ago
e52fd0b405
ck_fifo: Migrate MPMC FIFO to ck_pr_fence_X_Y.
12 years ago
0d827b4c81
ck_hp_fifo: Finer-grained mapping to ck_pr_fence_X_Y.
...
Specifically, atomic-RMW fences were previously generalized
as store operations.
12 years ago
babae24a07
ck_rwcohort: Added missing return statements and changed while (true) loops to for (;;)
12 years ago
2b33d57148
ck_rwcohort: Removed extra whitespace
12 years ago
cbfa095108
Merge branch 'master' of https://github.com/sbahra/ck
12 years ago
2ba3f59374
ck_spinlock: Migrate MCS to ck_pr_fence_X_Y.
...
This includes fixing acquire semantics on mcs_lock fast path.
This represents an additional fence on the fast path for
acquire semantics post-acquisition.
12 years ago
8540821f3f
ck_spinlock: Minor style changes to return statement.
12 years ago
08d13deaf4
ck_brlock: Migrate to ck_pr_fence_X_Y.
12 years ago
214d7aed66
ck_pr: Implement ck_pr_fence_atomic in MD ck_pr.
12 years ago
8320a4a6f4
ck_pr: Comment elaboration on ck_pr_fence_atomic semantics.
...
More specifically, note that in memory models where atomic
operations do not have serializing effects that atomic
read-modify-write operations are modeled as store operations.
12 years ago
3f06a4e23a
ck_rwlock: Use ck_pr_fence_atomic.
12 years ago
ffd22e57b8
ck_bytelock: Use ck_pr_fence_atomic.
12 years ago
a4220f2377
ck_pr: Remove ck_pr_fence_load_depends from GCC port.
12 years ago
d1dd6611ac
ck_pr: Add ck_pr_fence_atomic interface.
...
These operations serialize atomic-RMW operations with respect
to each other, loads and stores. In addition to this, the
load_depends implementations have been removed.
12 years ago
8face51e0d
ck_spinlock: Acquire semantics for anderson spinlock.
12 years ago
f87e0caf99
ck_bytelock: Migrate to ck_pr_fence_X_Y.
12 years ago
83bc7f9f54
ck_epoch: Migrate to ck_pr_fence_X_Y.
12 years ago
fe7e5ac5b1
ck_hp_stack: Migrate to ck_pr_fence_X_Y functions.
12 years ago
01f89ee691
ck_hp_fifo: Migrate to ck_pr_fence_X_Y functions.
12 years ago
b025722fbd
ck_brlock: Migrate to ck_pr_fence_X_Y interface.
12 years ago
5506ad2744
ck_pr: Move ck_pr_barrier to compiler port.
12 years ago
cc8c3fb2db
ck_pr/gcc: Fix style issue.
12 years ago
8311e9fcb4
ck_pr: Update GCC port and change ck_pr dependency path.
...
The compiler-specific ck_pr is now unconditionally included.
It currently implements things like compiler barriers.
12 years ago
44b769963f
ck_pr: ck_pr_fence_X_Y interface has been added.
...
ck_pr_fence_{load_load,store_store,load_store,store_load} operations
have been added. In addition to this, it is no longer the responsibility
of architecture ports to determine when to emit a specific fence. Instead,
the underlying port will always emit the necessary instructions to
enforce strict ordering. The higher-level include/ck_pr implementation will
enforce whether or not a fence is necessary to be emitted according to
the memory model specified by ck_md (CK_MD_{TSO,RMO,PSO}).
In other words, only ck_pr_fence_strict_* is implemented by the MD-ck_pr
port.
12 years ago
dcdc6dd100
ck_rwcohort: Fixed indentation and looping logic and added initial documentation
12 years ago
444df7dfe5
Merge branch 'master' of https://github.com/sbahra/ck
12 years ago
855a1472a8
ck_ht: Add ck_ht_reset_size_spmc operation.
...
The ck_ht_reset_size_spmc operation takes an additional argument
that allows the user to specify the size of the new generation
of the hashtable.
12 years ago
73531e9e48
ck_hs: Add ck_hs_reset_size operation.
...
The ck_hs_reset_size operation takes an additional argument
that allows the user to specify the size of the new generation
of the hashset.
12 years ago
4132ec4998
ck_epoch: Add ck_epoch_reclaim function.
...
This function allows for explicit execution of all
deferred callbacks in an epoch_record. The primary
motivation is currently for performance profiling
but there are other use-cases where best-effort
semantics could be applied.
12 years ago
cc4b248bcb
ck_rwcohort: Renamed ck_rwcohort namespace
12 years ago
e0f46e6045
merged from upstream repo
12 years ago
0d20391563
ck_rw_cohort: Added benchmark tests for reader-writer cohort locks
12 years ago
f693a16253
ck_rw_cohort: Added neutral reader-writer lock
12 years ago
21750b9321
ck_rw_cohort: Added read-preference logic and updated validation test to use it
12 years ago
1201e09577
ck_ring: Remove stale comment.
12 years ago
f93369c4fc
ck_rw_cohort: Removed backwards jump
12 years ago
a352b46d0b
ck_rw_cohort: renamed ck_rw_cohort namespace
12 years ago
011a9751a6
ck_queue: Add traversal-safe SPMC STAILQ.
12 years ago
5a13fddf70
ck_hp_fifo: Style fixes.
12 years ago
7980489eaf
ck_hp_stack: Style fixes.
12 years ago
5c483e3b1c
ck_hp: hp_acquire patterns require strict fences under TSO.
...
Several counter-examples were found which break in the
presence of store-to-load re-ordering. Strict fence
semantics are necessary.
Thanks to Paul McKenney for helpful discussions.
12 years ago
e827557ee9
ck_hs: Whitespace change.
12 years ago
d6fefccd6e
ck_pflock: Update comment, implementation is not so naive now.
12 years ago
676f8fed88
Merge branch 'cohort_trylock' of git://github.com/bscheinman/ck
12 years ago
878b8e6321
ck_ring: Correct CK_RING_SPMC_ENQUEUE_SIZE typo.
12 years ago
67381af3c5
ck_ring: Expose *_size variants to type-specialized ck_ring.
...
Added:
- CK_RING_ENQUEUE_SPSC_SIZE
- CK_RING_ENQUEUE_SPMC_SIZE
12 years ago
45f648bd33
ck_ring: Add *_size_* ck_ring enqueue operations.
...
These variants of ck_ring_enqueue_* return the snapshot of queue
length with respect to the linearization point. This can be used to
extract ring size without incurring additional cacheline invalidation
overhead from the writer.
12 years ago
646cb2cb06
ck_cohort_rw: Removed unused member variable
12 years ago
86884ed574
ck_fifo_mpmc: Fix NULL deference when re-using nodes.
...
If a FIFO entry is being re-used, it is possible for NULL
assignment to be triggered due to race with enqueue.
12 years ago
c4e50a193a
ck_fifo: Style fixes.
12 years ago
63bfa0d353
ck_cohort_rw: Reduced line wrapping
12 years ago
5e1b614108
ck_cohort_rw: Initial implementation with validation test.
...
I still need to implement benchmark tests and write documentation. The reader-writer cohort locks also required that I add a method to the existing ck_cohort framework to determine whether or not a cohort lock is currently in a locked state.
12 years ago
6441c90cfe
Merge branch 'master' of https://github.com/sbahra/ck
12 years ago
3c8728b214
ck_cohort: Added trylock support to cohort framework
12 years ago
4b9badd440
ck_cohort: Remove unnecessary whitespace.
12 years ago
62c121f500
ck_cohort: Added example code to man page
12 years ago
c393a97cb7
ck_pflock: Fix Copyright typo from original patch.
12 years ago
f6a53fd9de
ck_cohort: First-cut implementation of generalized cohort interface.
...
This work was contributed by Brendon Scheinman. This work is based
off "Lock Cohorting: A General Technique for Designing NUMA Locks".
12 years ago
060b922dc2
ck_pflock: Whitespace change.
...
Move acquire semantics comment block on read_unlock beneath
exit label.
12 years ago
aec2797d27
ck_pflock: Load acquire semantics are sufficient for read_unlock.
...
Also, make sure to implement acquire semantics if there are no
contending writers.
12 years ago
54845bb8e7
ck_pflock: Phase-fair read-write lock implementation.
...
John Wittrock has contributed a phase-fair reader-writer
lock implementation. These locks allow phase fairness
guarantees between readers and writers. This work includes
additional changes and clean-up.
Follow-up work is expected.
Thanks to John Wittrock for patches and Professor Gabriel
Parmer (http://www.seas.gwu.edu/~gparmer/ ) for advising.
12 years ago
7d3fd9d227
ck_fifo: Get rid of load_depends usage for now.
...
Until a target pops up with a requirement for load_depends(),
the cost is not worth it for now.
12 years ago
b7d258b219
ck_pr/gcc: ck_pr_fence_load_depends should be heavy-weight on catch-all target.
12 years ago
d09e48daed
ck_spinlock: Only enable trylock path for TSO architectures at the moment.
12 years ago
27d454248d
ck_spinlock: Add ticket lock with trylock operation.
...
Upon popular request, added a variant of the ticket spinlock
with trylock support. This is pending additional verification
on other architectures besides x86*. It is still unclear whether
this implementation will be the default as it is has slower
fast path.
Add trylock support to the ck_spinlock validation tests.
It currently only tests ck_spinlock_ticket_t trylock
functionality if available.
12 years ago
086ebf7126
ck_pr: moving PPC32 to RMO.
...
Signed-off-by: Joao Fernandes <mail@joaofernandes.eu>
12 years ago
8c53de1e70
ck_rwlock: Add documentation on visibility semantics.
12 years ago
1ea5769fd4
ck_rwlock: Improve code legibility.
12 years ago
334b15e6e0
ck_bytelock: Acquire semantics on read.
12 years ago
038b3d816d
ck_rwlock: Forgot to add acquire on operation success.
12 years ago
0598307186
whitespace: Various style changes.
12 years ago
25658c4f5f
ck_queue: Fix CK_LIST_INSERT_HEAD.
...
CK_LIST_INSERT_HEAD was incorrectly managing prev
pointer on insertion to non-empty list. This bug
would cause erroneous behavior on CK_LIST_REMOVE
to non-head elements. Unit test will be updated
for this regression.
12 years ago
33a9222923
legal: Update Copyright statements.
12 years ago
d29ac5afdb
ck_bytelock: Fix downgrade path from writer to reader.
...
An off-by-one was introduced in downgrade path from writer.
This can cause deadlock if a writer downgrades from a write lock.
Pointed out by Jeffrey Birnbaum <jmb...@...>.
12 years ago
c4149c72b4
ck_ring: Use regular assignment in type specialized ck_ring initialization.
12 years ago
89cb712ffa
ck_ring: Memoize mask across all operations.
12 years ago
8519aaf771
ck_ring: Memoize mask for SPMC operations.
12 years ago
b54ae5c4ac
ck_ring: Add a work-around for compiler bug(s) in ck_ring_dequeue_spmc.
...
Both LLVM-backed compilers and GCC incorrectly treat
a barrier-sandwiched load as a loop invariant in dequeue_spmc.
Forcing volatile atomic load semantics generates the right
thing.
Thanks to Devon O'Dell and Abel Mathew for help in catching
this issue.
12 years ago
0631e396e0
ck_backoff: Replace ck_backoff_eb with ck_backoff_gb.
...
The distinction between additive/exponential implementation
and geometric implementation does little but confuse users.
The terminology used in ck_backoff now reflects terminology
used in literature.
ck_backoff_gb has been removed.
12 years ago
b37ce19976
ck_pr/x86_64: GCC will attempt immediate expansion on double.
...
Break out double functionality to not use immediate expansion
at all.
Pointed out by Abel Mathew <amathew@appnexus...>.
12 years ago
0c56b1b8c6
ck_epoch: Remember to mark entry availability on unregister.
...
Pointed out by Matt Johnson (johnso87@crhc....>.
12 years ago
93684f77c1
ck_pr: Use CK_CC_INLINE instead of inline keyword for ck_pr_barrier.
12 years ago
de0597d2aa
ck_bitmap: Add comment for ck_bitmap_size.
12 years ago
849f938973
ck_ring: Add parameterized version of ck_ring_trydequeue.
12 years ago
7a99585f9e
ck_ring: No need for volatile load on dequeue_spmc.
12 years ago
01dc51db65
ck_ring: Add trydequeue variant.
12 years ago
ca8be0858c
ck_pr: Re-include port selection logic.
12 years ago
1c7b8811c8
ck_pr/gcc: Remove useless assignment from volatile load.
12 years ago
a9ed19303f
ck_pr/gcc: Adopt const semantics for load/store in GCC port.
12 years ago
6de54d6aa5
ck_pr/x86_64: Use correct target for load_64_2.
12 years ago
12da4128ff
ck_pr: Adopt const qualifer for load/store.
...
We will adopt these semantics for the rest of _ptr
family at some point in the future as well.
12 years ago
055a7fd5db
ck_ring: Add wrapper functions for parameterized SPMC ring.
12 years ago
68b33e56d8
ck_ring: Add parameterized version of SPMC ring.
12 years ago
7f35cbee1b
ck_ring: Use SPSC enqueue as SPMC enqueue.
12 years ago
5d1d850f44
ck_ring: Adopt semantics identical to spmc counter semantics.
12 years ago
77a3bd3126
ck_ring: Move overflow check to producer side.
...
unsigned int-sized counters will be used on producer
and consumer side. This is necessary to allow a ring
to be used on multiple workloads.
12 years ago
d595bafea7
ck_ring: Serialize producer snapshot with respect to consumer.
12 years ago
d705e02b76
ck_ring: Replace MPMC variant with SPMC variant for starters.
...
Immediate use-case is SPMC. Next is MPSC, then I will generalize
to MPMC (unfortunately, I don't have an algorithm that avoids CAS2
yet for MPMC).
12 years ago
bb48f602ff
ck_ring: Add note regarding correctness so people don't both verifying it.
...
Original version is susceptible to several problems, it is whack.
12 years ago
68e6752e53
ck_ring: Due to popular request, add MPMC variants of enqueue/dequeue.
...
Fencing semantics are heavier-weight than they need to be.
12 years ago
957e4aec5d
ck_brlock: Fix typo.
12 years ago
b631e3aa23
ck_brlock: Enable FAS serialization only on x86.
...
Other architectures, even TSO ones, pay a heavier flush
cost.
12 years ago
2bdc231288
ck_brlock: Serialize on atomic operation on read acquisition.
...
10% improvement for reader acquisition latency on TSO architectures
implementing wait-free FAS.
12 years ago
cb1c72d36b
ck_queue: Add CK_?LIST_MOVE operation.
...
This operation is of format:
CK_S*LIST_MOVE(a, b, linkage) and is equivalent to intializing
a with the contents of b. This is done in a manner that is atomic
with respect to readers. Read-only operations are still valid in
b, but behavior is undefined for write-side operations on b after
a MOVE operation.
12 years ago
5df8882840
ck_bytelock: Switch to full semantics on unlock.
...
RW semantics may be used for shared write-side.
12 years ago
a06f28b77c
ck_ring: Remove useless comment.
12 years ago
f3111b006b
ck_fifo: SPSC FIFO dequeue only requires load depends barrier.
12 years ago
bf30d3439c
ck_rwlock: Remove unncessary load fences, switch to full semantics on read unlock.
...
rwlock may also be used in SPMC workloads.
12 years ago
65ebf86799
ck_spinlock: Switch to full semantics across all implementations.
...
More importantly, trylock previously didn't have full semantics.
12 years ago
e40521d1aa
ck_pr/ppc64: Switch to RMO model for atomic operations.
...
Signed-off-by: Samy Al Bahra <sbahra@repnop.org>
12 years ago
6be03bb450
ck_bytelock: Remove unnecessary load barrier on unslotted read acquisition.
...
Signed-off-by: Samy Al Bahra <sbahra@repnop.org>
12 years ago
52ca64b2b7
ck_pr/sparcv9: MemIssue is overly heavyweight.
...
Signed-off-by: Samy Al Bahra <sbahra@repnop.org>
12 years ago
3f217c9789
ck_pr: Fallback to RMO for PSO for this release.
...
Barriers can be rejiggered next release.
12 years ago
a1dc38f20e
build/ck_pr: Add configurable memory models.
12 years ago
fedc2f6e47
ck_pr/sparcv9: Assume TSO by default.
12 years ago
690782aa67
ck_hs: Add hash function wrapper to maintain opacity of ck_hs_t.
...
This is in the form of CK_HS_HASH(X, Y, Z) which expands to
Y(Z, X.seed).
12 years ago
f2dfd828c3
Merge pull request #7 from abelmathew/master
...
CK_MD_VMA_BITS
12 years ago
ece2895e9e
ck_hs: Drop ck_hs_hash_t abstraction.
...
No need for opaque type anymore.
12 years ago
00d0320767
ck_bag: Fix usage of CK_MD_VMA_BITS.
12 years ago
adaf20b31c
ck_ht: Add ck_ht_stat function.
...
This can be used to expose certain performance metrics.
12 years ago
0f5e540afa
ck_hs: Lock-free set loosely modeled after ck_ht.
...
Currently only SPMC, but MPMC transformation is relatively trivial.
Documentation will come in a follow-up commit.
12 years ago
ec02b2f356
ck_bag: USE CK_BAG_KEY_LENGTH
12 years ago
9ff3c2d017
Merge remote-tracking branch 'upstream/master'
12 years ago
981e0a49a2
ck_bag: Use CK_MD_VMA_BITS
12 years ago
1d1e433dc4
ck_ht: Set unsigned key length.
12 years ago
ec0d212c04
ck_ht: Set artificial key length limit if pointer packing is disabled.
12 years ago
90713046e9
ck_ht: Use CK_MD_VMA_BITS.
...
Documentation update is necessary to point out CK_HT_KEY_LENGTH.
12 years ago
7a04c368ff
build: Add VMA bits detection for Mac OS X and Linux on x86*.
12 years ago
c45098b21b
ck_epoch: Emit fence instructions for TSO, do not emit for recursive calls.
...
Unfortunately, there are still workloads which will not
serialize. Recursive calls into ck_epoch_begin have no
need for fence instructions.
12 years ago
a5fb24ab61
ck_ht: Hard-code ck_ht_entry alignment.
...
Even though there is a re-write in progress, this issue must
be fixed. The CK_CC_ALIGNED attribute was never added.
12 years ago
f6a675ab00
ck_cc: Add no-op macros in case of unsupported compiler.
12 years ago
57104fcde7
ck_epoch: Change to epoch semantics, bump grace period to 4.
12 years ago
Samy Al Bahra
Samy Al Bahra
User Doginou
Olivier Houchard
Brendon Scheinman
Joao Fernandes
Abel Mathew