- ck_epoch_begin: Disallow early load of epoch as it leads to measurable
performance degradation in some benchmarks.
- ck_epoch_synchronize: Enforce barrier semantics.
This inefficiency was introduced in the overhaul of
the ck_epoch API.
Synchronize is executed with respect to e. At e + 1,
references can only exist to objects logically deleted
at e or e + 1. At e + 2, however, references can only exist to
objects logically deleted at e + 1 and e + 2. In the case that a
thread observes an out of date epoch value, an increment to the
global epoch would fail as the active bit is ordered with respect
to the memory barrier in synchronize. In the case that a protected
section begins after the memory barrier, then it is guaranteed
to not acquire the hazardous reference.
This does not change granularity of deferral lists, however.
There is still a requirement of 3 deferral lists on the fast path
(4 in ck_epoch for fast path purposes) as at any moment, any given
deferral list for value e can contain references to objects with
active references from both e and e - 1.
This function allows for explicit execution of all
deferred callbacks in an epoch_record. The primary
motivation is currently for performance profiling
but there are other use-cases where best-effort
semantics could be applied.
This allows us to only wait two rather than three epoch-protected
sections, which is a considerable performance improvement. Thanks
much to Richard Bornat for clarifications and feedback.
I had the pleasure of spending a significant amount of time at the most
recent LPC with Mathieu Desnoyers and Paul McKenney. In discussing
RCU semantics in relation to epoch reclamation, it was argued that
epoch reclamation is a specialisation of RCU (rather than a generalization).
In light of this discussion, I thought it would make more sense to not expose
write-side synchronization semantics aside from ck_epoch_call (similar to
RCU call), ck_epoch_poll (identical to tick), ck_epoch_barrier and
ck_epoch_synchronization (similar to ck_epoch_synchronization). Writers will
now longer have to use write-side epoch sections but can instead rely on
epoch_barrier/synchronization for blocking semantics and ck_epoch_poll
for old tick semantics.
One advantage of this is we can avoid write-side recursion for certain workloads.
Additionally, for infrequent writes, epoch_barrier and epoch_synchronization both
allow for blocking semantics to be used so you don't have to pay the cost of
epoch_entry for non-blocking dispatch.
Example usage:
e = stack_pop(mystack);
ck_epoch_synchronize(...);
free(e);
read_begin and read_end has been replaced with ck_epoch_begin and ck_epoch_end.
If multiple writers need SMR guarantees, then they can also use ck_epoch_begin
and ck_epoch_end. Any dispatch in presence of multiple writers should be done
with-in an epoch section (for now).
There are some follow-up commits to come.
These changes primarily affect platforms with RMO semantics.
- Write out to local epoch requires atomic semantics,
adopt atomic store to epoch in ck_epoch_reclaim.
- Correct an issue where we would publish node availability
before record re-initialization.
- Execute full barrier in write_begin to serialize active flag
IFF we are not recursing into an epoch block. Serialize active
publication with respect to epoch walk.
- Use CAS-based tick operation for epoch counter. Despite higher
constant factor, we are much less likely to starve writers on
bursty workloads.
Samy Al Bahra
Olivier Houchard
Samy Al Bahra
Samy Al Bahra