haiwan
/
ck
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
Branches Tags
${ item.name }
Create tag ${ searchTerm }
Create branch ${ searchTerm }
from 'master'
${ noResults }
ck/include/gcc/sparcv9/ck_pr.h

249 lines
6.6 KiB
Raw Permalink Normal View History Unescape

Initial import.
14 years ago
/*
* Copyright 2009, 2010 Samy Al Bahra.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*/
whitespace: Strictly conform to C namespacing rules.
10 years ago
#ifndef CK_PR_SPARCV9_H
#define CK_PR_SPARCV9_H
Initial import.
14 years ago
whitespace: Strictly conform to C namespacing rules.
10 years ago
#ifndef CK_PR_H
Initial import.
14 years ago
#error Do not include this file directly, use ck_pr.h
#endif
#include <ck_cc.h>
ck_pr/sparcv9: Assume TSO by default.
12 years ago
#include <ck_md.h>
Initial import.
14 years ago
/*
* The following represent supported atomic operations.
* These operations may be emulated.
*/
#include "ck_f_pr.h"
ck_pr: Update GCC port and change ck_pr dependency path. The compiler-specific ck_pr is now unconditionally included. It currently implements things like compiler barriers.
12 years ago
/*
* Minimum interface requirement met.
*/
#define CK_F_PR
Initial import.
14 years ago
/*
* Order loads at the least.
*/
CK_CC_INLINE static void
ck_pr_stall(void)
{
__asm__ __volatile__("membar #LoadLoad" ::: "memory");
return;
}
ck_pr: ck_pr_fence_X_Y interface has been added. ck_pr_fence_{load_load,store_store,load_store,store_load} operations have been added. In addition to this, it is no longer the responsibility of architecture ports to determine when to emit a specific fence. Instead, the underlying port will always emit the necessary instructions to enforce strict ordering. The higher-level include/ck_pr implementation will enforce whether or not a fence is necessary to be emitted according to the memory model specified by ck_md (CK_MD_{TSO,RMO,PSO}). In other words, only ck_pr_fence_strict_* is implemented by the MD-ck_pr port.
12 years ago
#define CK_PR_FENCE(T, I) \
CK_CC_INLINE static void \
ck_pr_fence_strict_##T(void) \
{ \
__asm__ __volatile__(I ::: "memory"); \
}
Initial import.
14 years ago
ck_pr: Add ck_pr_fence_atomic interface. These operations serialize atomic-RMW operations with respect to each other, loads and stores. In addition to this, the load_depends implementations have been removed.
12 years ago
/*
* Atomic operations are treated as both load and store
* operations on SPARCv9.
*/
ck_pr: Implement ck_pr_fence_atomic in MD ck_pr.
12 years ago
CK_PR_FENCE(atomic, "membar #StoreStore")
ck_pr: Add ck_pr_fence_atomic interface. These operations serialize atomic-RMW operations with respect to each other, loads and stores. In addition to this, the load_depends implementations have been removed.
12 years ago
CK_PR_FENCE(atomic_store, "membar #StoreStore")
CK_PR_FENCE(atomic_load, "membar #StoreLoad")
CK_PR_FENCE(store_atomic, "membar #StoreStore")
CK_PR_FENCE(load_atomic, "membar #LoadStore")
ck_pr/sparcv9: Recent SPARCs actually implement RSO. ck_pr_fence_store/load will enforce store/load barriers by default from now on.
14 years ago
CK_PR_FENCE(store, "membar #StoreStore")
ck_pr: ck_pr_fence_X_Y interface has been added. ck_pr_fence_{load_load,store_store,load_store,store_load} operations have been added. In addition to this, it is no longer the responsibility of architecture ports to determine when to emit a specific fence. Instead, the underlying port will always emit the necessary instructions to enforce strict ordering. The higher-level include/ck_pr implementation will enforce whether or not a fence is necessary to be emitted according to the memory model specified by ck_md (CK_MD_{TSO,RMO,PSO}). In other words, only ck_pr_fence_strict_* is implemented by the MD-ck_pr port.
12 years ago
CK_PR_FENCE(store_load, "membar #StoreLoad")
ck_pr/sparcv9: Recent SPARCs actually implement RSO. ck_pr_fence_store/load will enforce store/load barriers by default from now on.
14 years ago
CK_PR_FENCE(load, "membar #LoadLoad")
ck_pr: ck_pr_fence_X_Y interface has been added. ck_pr_fence_{load_load,store_store,load_store,store_load} operations have been added. In addition to this, it is no longer the responsibility of architecture ports to determine when to emit a specific fence. Instead, the underlying port will always emit the necessary instructions to enforce strict ordering. The higher-level include/ck_pr implementation will enforce whether or not a fence is necessary to be emitted according to the memory model specified by ck_md (CK_MD_{TSO,RMO,PSO}). In other words, only ck_pr_fence_strict_* is implemented by the MD-ck_pr port.
12 years ago
CK_PR_FENCE(load_store, "membar #LoadStore")
ck_pr/sparcv9: use the more stringent #MemIssue barrier. This is to match the semantics of the FreeBSD kernel.
7 years ago
CK_PR_FENCE(memory, "membar #MemIssue")
ck_pr: Add acquire and release fences.
11 years ago
CK_PR_FENCE(acquire, "membar #LoadLoad | #LoadStore")
CK_PR_FENCE(release, "membar #LoadStore | #StoreStore")
ck_pr: Add ck_pr_fence_acqrel. Provides both acquire and release semantics.
9 years ago
CK_PR_FENCE(acqrel, "membar #LoadLoad | #LoadStore | #StoreStore")
ck_pr: Introduce ck_pr_fence_lock and fence_unlock. These primitives are meant to be used in lock implementations where control dependency ordering is sufficient to enforce ordering of critical section. At the moment, this only affects PPC. Currently, we rely on lwsync for entry into critical sections which is insufficient. sync is rather heavy-weight, and assuming we aren't falling victim into compiler re-ordering, isync should be sufficient. There is follow-up work to be done in ARM, as we may have cheaper (but target-specialized) ISB-tricks for load-load ordering.
10 years ago
CK_PR_FENCE(lock, "membar #LoadLoad | #LoadStore | #StoreStore | #StoreLoad")
CK_PR_FENCE(unlock, "membar #LoadStore | #StoreStore")
Initial import.
14 years ago
#undef CK_PR_FENCE
#define CK_PR_LOAD(S, M, T, C, I) \
CK_CC_INLINE static T \
Make ck_pr_store_* and ck_pr_load_* a bit more type safe. We use some macro trickery to enforce that ck_pr_store_* is actually storing the correct type into the target variable, without any actual side effects--by making the assignment into an rvalue and using a comma expression, the compiler should optimize it away. On the load side, we simply cast the result to the type of the target variable for pointer loads. There is an unsafe version of the store_ptr macro called ck_pr_store_ptr_unsafe for those times when you are _really_ sure that you know what you're doing. This commit also updates some of the source files (ck_ht, ck_hs, ck_rhs): ck_ht now uses the unsafe macro, as its conversion between uintptr_t and void * is invalid under the new macros. ck_hs and ck_rhs have had some casts added to preserve validity.
10 years ago
ck_pr_md_load_##S(const M *target) \
Initial import.
14 years ago
{ \
T r; \
__asm__ __volatile__(I " [%1], %0" \
: "=&r" (r) \
: "r" (target) \
: "memory"); \
return (r); \
}
CK_PR_LOAD(ptr, void, void *, uint64_t, "ldx")
#define CK_PR_LOAD_S(S, T, I) CK_PR_LOAD(S, T, T, T, I)
CK_PR_LOAD_S(64, uint64_t, "ldx")
CK_PR_LOAD_S(32, uint32_t, "lduw")
CK_PR_LOAD_S(uint, unsigned int, "lduw")
ck_pr: Add ck_pr_load_double()/ck_pr_store_double() in case volatile loads and stores of floating point values are needed.
13 years ago
CK_PR_LOAD_S(double, double, "ldx")
Initial import.
14 years ago
CK_PR_LOAD_S(int, int, "ldsw")
#undef CK_PR_LOAD_S
#undef CK_PR_LOAD
#define CK_PR_STORE(S, M, T, C, I) \
CK_CC_INLINE static void \
Make ck_pr_store_* and ck_pr_load_* a bit more type safe. We use some macro trickery to enforce that ck_pr_store_* is actually storing the correct type into the target variable, without any actual side effects--by making the assignment into an rvalue and using a comma expression, the compiler should optimize it away. On the load side, we simply cast the result to the type of the target variable for pointer loads. There is an unsafe version of the store_ptr macro called ck_pr_store_ptr_unsafe for those times when you are _really_ sure that you know what you're doing. This commit also updates some of the source files (ck_ht, ck_hs, ck_rhs): ck_ht now uses the unsafe macro, as its conversion between uintptr_t and void * is invalid under the new macros. ck_hs and ck_rhs have had some casts added to preserve validity.
10 years ago
ck_pr_md_store_##S(M *target, T v) \
Initial import.
14 years ago
{ \
__asm__ __volatile__(I " %0, [%1]" \
: \
: "r" (v), \
"r" (target) \
: "memory"); \
return; \
}
ck_pr: Adopt const qualifer for load/store. We will adopt these semantics for the rest of _ptr family at some point in the future as well.
12 years ago
CK_PR_STORE(ptr, void, const void *, uint64_t, "stx")
Initial import.
14 years ago
#define CK_PR_STORE_S(S, T, I) CK_PR_STORE(S, T, T, T, I)
CK_PR_STORE_S(8, uint8_t, "stub")
CK_PR_STORE_S(64, uint64_t, "stx")
CK_PR_STORE_S(32, uint32_t, "stuw")
CK_PR_STORE_S(uint, unsigned int, "stuw")
ck_pr: Add ck_pr_load_double()/ck_pr_store_double() in case volatile loads and stores of floating point values are needed.
13 years ago
CK_PR_STORE_S(double, double, "stx")
Initial import.
14 years ago
CK_PR_STORE_S(int, int, "stsw")
#undef CK_PR_STORE_S
#undef CK_PR_STORE
ck_pr/sparcv9: use the correct address space for atomics on FreeBSD On FreeBSD, atomic operations in the kernel must access the nucleus address space. Userland may use either the atomic instruction set which goes without an ASI (address space identifier) or specify the primary address space. To avoid hardcoding the address space here, we grab the corresponding identifier from the appropriate machine header but also only for the kernel so the namespace doesn't get polluted for userland.
6 years ago
/* Use the appropriate address space for atomics within the FreeBSD kernel. */
#if defined(__FreeBSD__) && defined(_KERNEL)
#include <sys/cdefs.h>
#include <machine/atomic.h>
#define CK_PR_INS_CAS "casa"
#define CK_PR_INS_CASX "casxa"
#define CK_PR_INS_SWAP "swapa"
#define CK_PR_ASI_ATOMIC __XSTRING(__ASI_ATOMIC)
#else
#define CK_PR_INS_CAS "cas"
#define CK_PR_INS_CASX "casx"
#define CK_PR_INS_SWAP "swap"
#define CK_PR_ASI_ATOMIC ""
#endif
Initial import.
14 years ago
CK_CC_INLINE static bool
ck_pr_cas_64_value(uint64_t *target, uint64_t compare, uint64_t set, uint64_t *value)
{
ck_pr/sparcv9: use the correct address space for atomics on FreeBSD On FreeBSD, atomic operations in the kernel must access the nucleus address space. Userland may use either the atomic instruction set which goes without an ASI (address space identifier) or specify the primary address space. To avoid hardcoding the address space here, we grab the corresponding identifier from the appropriate machine header but also only for the kernel so the namespace doesn't get polluted for userland.
6 years ago
__asm__ __volatile__(CK_PR_INS_CASX " [%1] " CK_PR_ASI_ATOMIC ", %2, %0"
Initial import.
14 years ago
: "+&r" (set)
: "r" (target),
"r" (compare)
: "memory");
*value = set;
return (compare == set);
all: Strip trailing whitespaces.
13 years ago
}
Initial import.
14 years ago
CK_CC_INLINE static bool
ck_pr_cas_64(uint64_t *target, uint64_t compare, uint64_t set)
{
ck_pr/sparcv9: use the correct address space for atomics on FreeBSD On FreeBSD, atomic operations in the kernel must access the nucleus address space. Userland may use either the atomic instruction set which goes without an ASI (address space identifier) or specify the primary address space. To avoid hardcoding the address space here, we grab the corresponding identifier from the appropriate machine header but also only for the kernel so the namespace doesn't get polluted for userland.
6 years ago
__asm__ __volatile__(CK_PR_INS_CASX " [%1] " CK_PR_ASI_ATOMIC ", %2, %0"
all: Strip trailing whitespaces.
13 years ago
: "+&r" (set)
Initial import.
14 years ago
: "r" (target),
"r" (compare)
: "memory");
return (compare == set);
}
CK_CC_INLINE static bool
ck_pr_cas_ptr(void *target, void *compare, void *set)
{
return ck_pr_cas_64(target, (uint64_t)compare, (uint64_t)set);
}
CK_CC_INLINE static bool
ck_pr_cas_ptr_value(void *target, void *compare, void *set, void *previous)
{
return ck_pr_cas_64_value(target, (uint64_t)compare, (uint64_t)set, previous);
all: Strip trailing whitespaces.
13 years ago
}
Initial import.
14 years ago
#define CK_PR_CAS(N, T) \
CK_CC_INLINE static bool \
ck_pr_cas_##N##_value(T *target, T compare, T set, T *value) \
{ \
ck_pr/sparcv9: use the correct address space for atomics on FreeBSD On FreeBSD, atomic operations in the kernel must access the nucleus address space. Userland may use either the atomic instruction set which goes without an ASI (address space identifier) or specify the primary address space. To avoid hardcoding the address space here, we grab the corresponding identifier from the appropriate machine header but also only for the kernel so the namespace doesn't get polluted for userland.
6 years ago
__asm__ __volatile__(CK_PR_INS_CAS " [%1] " CK_PR_ASI_ATOMIC ", %2, %0" \
Initial import.
14 years ago
: "+&r" (set) \
: "r" (target), \
"r" (compare) \
: "memory"); \
*value = set; \
return (compare == set); \
} \
CK_CC_INLINE static bool \
ck_pr_cas_##N(T *target, T compare, T set) \
{ \
ck_pr/sparcv9: use the correct address space for atomics on FreeBSD On FreeBSD, atomic operations in the kernel must access the nucleus address space. Userland may use either the atomic instruction set which goes without an ASI (address space identifier) or specify the primary address space. To avoid hardcoding the address space here, we grab the corresponding identifier from the appropriate machine header but also only for the kernel so the namespace doesn't get polluted for userland.
6 years ago
__asm__ __volatile__(CK_PR_INS_CAS " [%1] " CK_PR_ASI_ATOMIC ", %2, %0" \
Initial import.
14 years ago
: "+&r" (set) \
: "r" (target), \
"r" (compare) \
: "memory"); \
return (compare == set); \
}
CK_PR_CAS(32, uint32_t)
CK_PR_CAS(uint, unsigned int)
CK_PR_CAS(int, int)
#undef CK_PR_CAS
#define CK_PR_FAS(N, T) \
CK_CC_INLINE static T \
ck_pr_fas_##N(T *target, T update) \
{ \
\
ck_pr/sparcv9: use the correct address space for atomics on FreeBSD On FreeBSD, atomic operations in the kernel must access the nucleus address space. Userland may use either the atomic instruction set which goes without an ASI (address space identifier) or specify the primary address space. To avoid hardcoding the address space here, we grab the corresponding identifier from the appropriate machine header but also only for the kernel so the namespace doesn't get polluted for userland.
6 years ago
__asm__ __volatile__(CK_PR_INS_SWAP " [%1] " CK_PR_ASI_ATOMIC ", %0" \
Initial import.
14 years ago
: "+&r" (update) \
: "r" (target) \
: "memory"); \
return (update); \
}
CK_PR_FAS(int, int)
CK_PR_FAS(uint, unsigned int)
CK_PR_FAS(32, uint32_t)
#undef CK_PR_FAS
ck_pr/sparcv9: use the correct address space for atomics on FreeBSD On FreeBSD, atomic operations in the kernel must access the nucleus address space. Userland may use either the atomic instruction set which goes without an ASI (address space identifier) or specify the primary address space. To avoid hardcoding the address space here, we grab the corresponding identifier from the appropriate machine header but also only for the kernel so the namespace doesn't get polluted for userland.
6 years ago
#undef CK_PR_INS_CAS
#undef CK_PR_INS_CASX
#undef CK_PR_INS_SWAP
#undef CK_PR_ASI_ATOMIC
whitespace: Strictly conform to C namespacing rules.
10 years ago
#endif /* CK_PR_SPARCV9_H */
whitespace: Bulk whitespace changes.
11 years ago