We use some macro trickery to enforce that ck_pr_store_* is actually
storing the correct type into the target variable, without any actual
side effects--by making the assignment into an rvalue and using a
comma expression, the compiler should optimize it away.
On the load side, we simply cast the result to the type of the target
variable for pointer loads.
There is an unsafe version of the store_ptr macro called
ck_pr_store_ptr_unsafe for those times when you are _really_ sure that
you know what you're doing.
This commit also updates some of the source files (ck_ht, ck_hs,
ck_rhs): ck_ht now uses the unsafe macro, as its conversion between
uintptr_t and void * is invalid under the new macros. ck_hs and ck_rhs
have had some casts added to preserve validity.
Commit 554e2f08 removed underscores from _CK prefixes. It missed
the arm bits, breaking the arm build (which checks for the
non-existing _CK_PR_H). Fix it.
While at it, fix the mismatch between _CK_ISB and __CK_ISB; convert
them both to CK_ISB.
Signed-off-by: Emilio G. Cota <cota@braap.org>
This was accidentally grouped into previous commit.
The destiny of this interface for internal use is still unclear (in context of
utilization in built-in data structures). The interface is enabled by default
on x86, as it is compatible with read-side prefetch* operations and
binary-compatible with 3DNow! extension. Older compilers will waste an
additional byte (they generate 3DNow! variant) on this, but older compilers
waste more on spillage if we encode instruction. Power support is coming soon.
This has been on the TODO for a while and helps reduce
read-side retries. It also has the advantage of providing
true wait-freedom on insertion (including termination safety).
Some missing references found are not surprising (e.g. *_INITIALIZER),
but real mistakes are also found (ck_ht_init->ck_entry_set_direct).
ck/doc$ make refcheck
ck_brlock: ref to missing ck_bytelock(3)
ck_ht_get_spmc: ref to missing ck_ht_entry_set_key(3)
ck_ht_init: ref to missing ck_entry_set_direct(3)
ck_ht_remove_spmc: ref to missing ck_ht_entry_set_key(3)
ck_bitmap_test: ref to missing ck_bitmap_set_mpmc(3)
ck_bitmap_test: ref to missing ck_bitmap_reset_mpmc(3)
ck_bitmap_size: ref to missing ck_bitmap_set_mpmc(3)
ck_bitmap_size: ref to missing ck_bitmap_reset_mpmc(3)
ck_rwcohort: ref to missing CK_RWCOHORT_STRATEGY_PROTOTYPE(3)
ck_rwcohort: ref to missing CK_COHORT_INITIALIZER(3)
ck_rwcohort: ref to missing CK_COHORT_LOCKED(3)
CK_RWCOHORT_INIT: ref to missing CK_RWCOHORT_TRYLOCK_PROTOTYPE(3)
CK_RWCOHORT_INIT: ref to missing CK_RWCOHORT_INITIALIZER(3)
CK_RWCOHORT_INIT: ref to missing CK_RWCOHORT_LOCK(3)
CK_RWCOHORT_INIT: ref to missing CK_RWCOHORT_UNLOCK(3)
CK_RWCOHORT_INIT: ref to missing CK_RWCOHORT_LOCKED(3)
CK_RWCOHORT_INIT: ref to missing CK_RWCOHORT_TRYLOCK(3)
CK_RWCOHORT_INSTANCE: ref to missing CK_RWCOHORT_TRYLOCK_PROTOTYPE(3)
CK_RWCOHORT_INSTANCE: ref to missing CK_RWCOHORT_INITIALIZER(3)
CK_RWCOHORT_INSTANCE: ref to missing CK_RWCOHORT_LOCK(3)
CK_RWCOHORT_INSTANCE: ref to missing CK_RWCOHORT_UNLOCK(3)
CK_RWCOHORT_INSTANCE: ref to missing CK_RWCOHORT_LOCKED(3)
CK_RWCOHORT_INSTANCE: ref to missing CK_RWCOHORT_TRYLOCK(3)
CK_RWCOHORT_PROTOTYPE: ref to missing CK_RWCOHORT_INITIALIZER(3)
CK_RWCOHORT_READ_LOCK: ref to missing CK_RWCOHORT_INITIALIZER(3)
CK_RWCOHORT_READ_UNLOCK: ref to missing CK_RWCOHORT_INITIALIZER(3)
CK_RWCOHORT_WRITE_LOCK: ref to missing CK_RWCOHORT_INITIALIZER(3)
CK_RWCOHORT_WRITE_UNLOCK: ref to missing CK_RWCOHORT_INITIALIZER(3)
ck_cohort: ref to missing CK_COHORT_LOCKED(3)
ck_cohort: ref to missing CK_COHORT_INITIALIZER(3)
ck_cohort: ref to missing CK_COHORT_LOCKED(3)
CK_COHORT_PROTOTYPE: ref to missing CK_COHORT_INITIALIZER(3)
CK_COHORT_PROTOTYPE: ref to missing CK_COHORT_LOCKED(3)
CK_COHORT_TRYLOCK_PROTOTYPE: ref to missing CK_COHORT_INITIALIZER(3)
CK_COHORT_TRYLOCK_PROTOTYPE: ref to missing CK_COHORT_LOCKED(3)
CK_COHORT_INSTANCE: ref to missing CK_COHORT_INITIALIZER(3)
CK_COHORT_INSTANCE: ref to missing CK_COHORT_LOCKED(3)
CK_COHORT_INIT: ref to missing CK_COHORT_INITIALIZER(3)
CK_COHORT_INIT: ref to missing CK_COHORT_LOCKED(3)
CK_COHORT_LOCK: ref to missing CK_COHORT_INITIALIZER(3)
CK_COHORT_LOCK: ref to missing CK_COHORT_LOCKED(3)
CK_COHORT_UNLOCK: ref to missing CK_COHORT_INITIALIZER(3)
CK_COHORT_UNLOCK: ref to missing CK_COHORT_LOCKED(3)
CK_COHORT_TRYLOCK: ref to missing CK_COHORT_INITIALIZER(3)
CK_COHORT_TRYLOCK: ref to missing CK_COHORT_LOCKED(3)
ck_sequence: ref to missing ck_bytelock(3)
Signed-off-by: Emilio G. Cota <cota@braap.org>
ck/doc$ make objcheck 1>/dev/null
ck_hs_put_unique is missing from OBJECTS
ck_rhs_put_unique is missing from OBJECTS
Signed-off-by: Emilio G. Cota <cota@braap.org>
The tombstone and version counter update invariant was not respected
in all necessary places.
- If a concurrent load operation is preempted after observing
the version counter and key field of a slot, then the slot is moved
and re-used by another key-value pair, the load operation would
observe an inconsistent pair without the relevant version counter
update.
- On RMO architectures, a store fence was missing on the delete path
(tombstone placement must always be followed by version counter
update).
John Wittrock
Samy Al Bahra
Olivier Houchard
Emilio G. Cota
Phil Sorber
Samy Al Bahra
Theo Schlossnagle