Retrieve more accurate CPU frequency for runtime (#366)

* apply a hack to get more accurate cpufreq

* apply a flag to the spinloop pause in the worker sched_idle
master
Emil 3 years ago committed by GitHub
parent f004d6c827
commit a2188b8bae
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23

@ -1,6 +1,7 @@
CFILES := src/*.c
INCLUDES := -Iinclude/
# fPIC = Position Independent Code, necessary for linking to relative addresses.
CFLAGS := -fPIC -O3 -flto
# Strips out calls to assert() and disables debuglog

@ -15,7 +15,7 @@ libsledge defines a ABI between the sledgert runtime and a \*.so shared library
A SLEdge \*.so serverless module is generated by the latter portion of the aWsm/SLEdge toolchain.
The first portion of the toolchain is responsible for compiling a source program into a WebAssembly module. This is handled by standard compilers capable of emitting WebAssembly.
The second portion of the toolchain is the aWsm compiler, which generates a \*.bc file with a well defined ABI
The second portion of the toolchain is the aWsm compiler, which generates a \*.bc file with a well defined ABI.
The third portion of the toolchain is the LLVM compiler, which ingests a \*.bc file emitted by aWsm and the libsledge static library, and emits a SLEdge \*.so serverless module.
## Architecture
@ -24,16 +24,12 @@ In order to reduce the overhead of calling sledgert functions, libsledge operate
The `sledge_abi__wasm_module_instance` structure includes the WebAssembly function table and the WebAssembly linear memory. This subset was selected because the author believes that use of function pointers and linear memory is frequent enough that LTO when compiling the \*.so file is beneficial.
All WebAssembly state
## WebAssembly Instruction Implementation
Here is a list of WebAssembly instructions that depend on symbols from libsledge, libc, or sledgert (via the SLEdge ABI).
### [Control Instructions](https://webassembly.github.io/spec/core/syntax/instructions.html#control-instructions)
The ABI includes the
| Instruction | aWsm ABI | libc Dependencies | SLEdge ABI |
| ------------- | ------------------------- | ------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| call_indirect | `get_function_from_table` | `stderr`, `fprintf` | `sledge_abi__current_wasm_module_instance.table`, `sledge_abi__wasm_trap_raise`, `WASM_TRAP_INVALID_INDEX`, `WASM_TRAP_MISMATCHED_TYPE` |
@ -115,13 +111,6 @@ The ABI includes the
| memory.size | `instruction_memory_size` | | `sledge_abi__current_wasm_module_instance.memory` |
| None | `initialize_region` | | `sledge_abi__current_wasm_module_instance.memory`, `sledge_abi__wasm_memory_initialize_region` |
Discussion:
- Should `instruction_memory_grow` be moved into sledgert? This would simplify the handling of the "cache" and generating a memory profile?
- Rename `sledge_abi__wasm_globals_*` to `sledge_abi__wasm_global_*`
- Implement Unsupported Numeric Instructions
- Should the wasm global table be accessed directly instead of via a runtime function?
- Should the Function Table be handled by the \*.so file or sledgert? Are function pointers really called that frequently?
# SLEdge \*.so Module Loading / Initialization
@ -136,27 +125,32 @@ The `sledgert` runtime is invoked with an argument containing the path to a JSON
The path to the JSON file is passed to `module_alloc_from_json`, which uses the Jasmine library to parse the JSON, performs validation, and passes the resulting specification to `module_alloc` for each module definition found. `module_alloc` allocated heap memory for a `struct module` and then calls `module_init`. `module_init` calls `sledge_abi_symbols_init`, which calls `dlopen` on the _.so file at the path specified in the JSON and then calls `dlsym` to resolve symbols within the _.so module.
`module.abi.initialize_globals` -> `SLEDGE_ABI__INITIALIZE_GLOBALS` -> `populate_globals`
`module.abi.initialize_memory`-> `SLEDGE_ABI__INITIALIZE_MEMORY` -> `populate_memory`
`module.abi.initialize_table` -> `SLEDGE_ABI__INITIALIZE_TABLE` -> `populate_table`
`module.abi.entrypoint` -> `SLEDGE_ABI__ENTRYPOINT` -> `wasmf__start`
`module.abi.starting_pages` -> `SLEDGE_ABI__STARTING_PAGES` -> `starting_pages`
`module.abi.max_pages` -> `SLEDGE_ABI__MAX_PAGES` -> `max_pages`
`module.abi.globals_len` -> `SLEDGE_ABI__GLOBALS_LEN` -> `globals_len`
- `module.abi.initialize_globals` -> `SLEDGE_ABI__INITIALIZE_GLOBALS` -> `populate_globals`
- `module.abi.initialize_memory`-> `SLEDGE_ABI__INITIALIZE_MEMORY` -> `populate_memory`
- `module.abi.initialize_table` -> `SLEDGE_ABI__INITIALIZE_TABLE` -> `populate_table`
- `module.abi.entrypoint` -> `SLEDGE_ABI__ENTRYPOINT` -> `wasmf__start`
- `module.abi.starting_pages` -> `SLEDGE_ABI__STARTING_PAGES` -> `starting_pages`
- `module.abi.max_pages` -> `SLEDGE_ABI__MAX_PAGES` -> `max_pages`
- `module.abi.globals_len` -> `SLEDGE_ABI__GLOBALS_LEN` -> `globals_len`
`module init` then calls `module.abi.initialize_table`, which populates the indirect function table with the actual functions. This is performed once during module initialization because this table does not actually vary between instances of a module.
`module_init` then calls `module.abi.initialize_table`, which populates the indirect function table with the actual functions. This is performed once during module initialization because this table does not actually vary between instances of a module.
# SLEdge \*.so Module Instantiation
When `sledgert` receives a request at the registered port specified in the JSON, it performs assorted allocation and initialization steps. The scheduler sets the expected ABI symbols and yields to `current_sandbox_start`, which immediately calls `current_sandbox_init`. This function initializes the associated runtime state and
When `sledgert` receives a request at the registered port specified in the JSON, it performs allocation and initialization steps. The scheduler sets the expected ABI symbols and yields to `current_sandbox_start`, which immediately calls `current_sandbox_init`. This function initializes the associated runtime state and
1. calls `module.abi.initialize_globals` for the current sandbox if not NULL. This is optional because the module might not have been built with the `--runtime-globals`, in which case runtime globals are not used at all. If not NULL, the globals are set in the table.
2. calls `module.abi.initialize_memory`, which copies regions into the linear memory
2. calls `module.abi.initialize_memory`, which copies segments into the linear memory
`current_sandbox_init` calls `wasi_context_init` to initialize the WASI context within the runtime.
`current_sandbox_init` returns to `current_sandbox_start`, which sets up wasm traps using `setjmp` and then calls `module.abi.entrypoint`
# Questions:
# Discussion (follow-up with Github issues):
- Should `sledge_abi__current_wasm_module_instance` be turned into a macro defined int the ABI header?
- Should `sledge_abi__current_wasm_module_instance` be turned into a macro defined int the ABI header? That way it'll be easier to change the ABI symbols (change once, applied everywhere).
- Should `instruction_memory_grow` be moved into sledgert? This would simplify the handling of the "cache" and generating a memory profile?
- Rename `sledge_abi__wasm_globals_*` to `sledge_abi__wasm_global_*`
- Implement Unsupported Numeric Instructions
- Should the wasm global table be accessed directly instead of via a runtime function? If we expose the wasm global table to libsledge, then we have worse ABI stability, but better performance.
- Should the Function Table be handled by the \*.so file or sledgert? Are function pointers really called that frequently?

@ -1,8 +1,6 @@
#include <stdint.h>
#include "sledge_abi.h"
// TODO: Validate uint32_t as return value;
uint32_t
wasi_snapshot_preview1_args_get(__wasi_size_t argv_retoffset, __wasi_size_t argv_buf_retoffset)
{

@ -37,6 +37,7 @@ enum RUNTIME_SIGALRM_HANDLER
extern pid_t runtime_pid;
extern bool runtime_preemption_enabled;
extern bool runtime_worker_spinloop_pause_enabled;
extern uint32_t runtime_processor_speed_MHz;
extern uint32_t runtime_quantum_us;
extern enum RUNTIME_SIGALRM_HANDLER runtime_sigalrm_handler;

@ -407,6 +407,9 @@ scheduler_idle_loop()
/* Clear the cleanup queue */
local_cleanup_queue_free();
/* Improve the performance of spin-wait loops (works only if preemptions enabled) */
if (runtime_worker_spinloop_pause_enabled) pause();
}
}

@ -38,8 +38,9 @@ uint32_t runtime_worker_threads_count = 0;
enum RUNTIME_SIGALRM_HANDLER runtime_sigalrm_handler = RUNTIME_SIGALRM_HANDLER_BROADCAST;
bool runtime_preemption_enabled = true;
uint32_t runtime_quantum_us = 5000; /* 5ms */
bool runtime_preemption_enabled = true;
bool runtime_worker_spinloop_pause_enabled = false;
uint32_t runtime_quantum_us = 5000; /* 5ms */
uint64_t runtime_boot_timestamp;
pid_t runtime_pid = 0;
@ -101,32 +102,48 @@ runtime_allocate_available_cores()
* We are also assuming this value is static
* @return proceccor speed in MHz
*/
static inline uint32_t
static inline void
runtime_get_processor_speed_MHz(void)
{
uint32_t return_value;
char *proc_mhz_raw = getenv("PROC_MHZ");
FILE *cmd = NULL;
FILE *cmd = popen("grep '^cpu MHz' /proc/cpuinfo | head -n 1 | awk '{print $4}'", "r");
if (unlikely(cmd == NULL)) goto err;
char buff[16];
size_t n = fread(buff, 1, sizeof(buff) - 1, cmd);
if (unlikely(n <= 0)) goto err;
buff[n] = '\0';
float processor_speed_MHz;
n = sscanf(buff, "%f", &processor_speed_MHz);
if (unlikely(n != 1)) goto err;
if (unlikely(processor_speed_MHz < 0)) goto err;
if (proc_mhz_raw != NULL) {
/* The case with manual override for the CPU freq */
runtime_processor_speed_MHz = atoi(proc_mhz_raw);
} else {
/* The case when we have to get CPU freq from */
usleep(200000); /* wait a bit for the workers to launch for more accuracy */
/* Get the average of the cpufreqs only for worker cores (no event core and reserved) */
char command[128] = { 0 };
sprintf(command, "grep '^cpu MHz' /proc/cpuinfo | sed -n '%u,%up' | \
awk '{ total += $4; count++ } END { print total/count }'",
runtime_first_worker_processor + 1,
runtime_first_worker_processor + runtime_worker_threads_count);
cmd = popen(command, "r");
if (unlikely(cmd == NULL)) goto err;
char buff[16];
size_t n = fread(buff, 1, sizeof(buff) - 1, cmd);
buff[n] = '\0';
float processor_speed_MHz;
n = sscanf(buff, "%f", &processor_speed_MHz);
if (unlikely(n != 1)) goto err;
if (unlikely(processor_speed_MHz < 0)) goto err;
runtime_processor_speed_MHz = (uint32_t)nearbyintf(processor_speed_MHz);
}
return_value = (uint32_t)nearbyintf(processor_speed_MHz);
pretty_print_key_value("Worker CPU Freq", "%u MHz\n", runtime_processor_speed_MHz);
done:
pclose(cmd);
return return_value;
return;
err:
return_value = 0;
goto done;
panic("Failed to detect processor frequency");
}
/**
@ -232,6 +249,17 @@ runtime_configure()
http_session_perf_log_init();
}
void
runtime_configure_worker_spinloop_pause()
{
/* Runtime Worker-Spinloop-Pause Toggle */
char *pause_enable = getenv("SLEDGE_SPINLOOP_PAUSE_ENABLED");
if (pause_enable != NULL && strcmp(pause_enable, "true") == 0) runtime_worker_spinloop_pause_enabled = true;
pretty_print_key_value("Worker-Spinloop-Pause", "%s\n",
runtime_worker_spinloop_pause_enabled ? PRETTY_PRINT_GREEN_ENABLED
: PRETTY_PRINT_RED_DISABLED);
}
void
log_compiletime_config()
{
@ -448,13 +476,6 @@ main(int argc, char **argv)
printf("Runtime Environment:\n");
runtime_processor_speed_MHz = runtime_get_processor_speed_MHz();
if (unlikely(runtime_processor_speed_MHz == 0)) panic("Failed to detect processor speed\n");
int heading_length = 30;
pretty_print_key_value("Processor Speed", "%u MHz\n", runtime_processor_speed_MHz);
runtime_set_resource_limits_to_max();
runtime_allocate_available_cores();
runtime_configure();
@ -463,6 +484,8 @@ main(int argc, char **argv)
listener_thread_initialize();
runtime_start_runtime_worker_threads();
runtime_get_processor_speed_MHz();
runtime_configure_worker_spinloop_pause();
software_interrupt_arm_timer();
#ifdef LOG_TENANT_LOADING

Loading…
Cancel
Save