What's going on with my software development focus
Aloha! The first question marks I left in a notebook was concerning the source of a "pool of threads" where a distributed computing model would from-draw. The context was in game development for an eventually-correct multiplayer simulation where there was no centralized server AND imperfect information sharing between connected computers. Regardless of hardware caps the main concern was that any data stored in memory would be interrogable by the host, meaning: if one player received an asyncronous update for another player's changes but the consequences of that change would by the game's rules not yet be visible to the greater environment, that update could be preperceived by the legal cracking of one's own device memory.
At the bare minimum any delayed messages would still need to be processed by some resource on the rendering machine in time for the results that SHOULD be shown perfectly, but that consistency did not need to be necessarily lived in the accessible components for that player so long as the computation could be verified once it reached the surface and the states synced. This implied a non-player cloud before general cloud was a thing, as well as a precedent for blockchain mining protocols dictating a software-level distribution of unidirectional compute sharding...at the base level though the effects of taking in batches of data at the same time as running an intensive gaming application would create an undesireable experience for both the CPU and the user as dynamic memory pressure inevitably leads to inefficient cases of object creation at runtime...
While I've done some GPU programming in the past, it is with the advent of "general purpose" GPU availabilities that I have found the ultimate answer to "pool of threads" in contexts including and beyond game development. My working title for this answer is GP2, where I focus on the most stable and vendor-unlocked avenues for non-graphics related code architecture and tooling. For the last decade I've been tackling the web platform with the foreknowledge that eventually other OS platforms would devolve into mere shells for HTML + JavaScript and with the release of WebGPU I am calling that deevolution complete!
A Problem Statement
A major issue with automating GPU work is similar to the early OS crashes: memory access flexibility led to conflicts, processor availability could stall time-sensitive operations and deadlock programs, programs written for one architecture would have differing functionality on newer editions, and fundamental resources needed intermediaring that did not yet exist. By preparing for generic memory management and multi-tasking with higher-level languageing which falls back to the lowest accepted baselines which do not assume an "outside world" when it comes to things like time or stdout that would otherwise be desiderata when preparing code to run in usefully-distributed ways, A Game Engine is simply the most demanding example of a full-stack cross-worlds environment conceivable! That is why I have always targeted game development: challenge
The Modules So Far, Introducing GP2 for the Web
GP2 Store is the primary module linking the GPU concepts of shaders, buffers and bindings to an agnostic management engine. While everything in the browser is required to be written in JavaScript at the time of this writing, the object of a Store is to bring everything into a data space in a way that can be compared and persisted. Because WebGPU is opinionatedly low-level and has several redundancies from a conceptual viewpoint the Store provides an API which blends most of what can be 1-to-1'd while still maintaining the raw connections to GPU land in order to attempt restartable/failproof consistency in case of a connection loss at worst or replication across devices where instantiation order of resources would matter most.
GP2 Shader is a compromise between the intentional limitations on WGSL and the requirements of dynamic code execution. RAM is more than a number, it is a method of access for a runtime that needs to switch focus and at the GPU level there exists well-defined strata of memory availability the optimizations for which cannot be deterministically compiled for without a planner that can pre-orchestrate between dispatches and workgroup caches. Depending on the reported capabilities of devices different logic must be constructed to achieve the same execution results - and while WebGPU provides the surface for that reporting and execution the experimentation of this tool suite revealed several gaps where greater flexibility could be coerced within the generic framework without losing the performance benefits of offloading of compute operations to GPU land.
The biggest TODO to date is the web component-ification of the separate JavaScript modules which all write to global window variables. Ideally state management would allow the consumption of multiple stores which unify the common objects of all related shaders and then events could be targeted through the shared backend of the custom element. Because GP2 is not web-specific this avenue has not been prioritized - the architecture of "many imports one interface" is the goal regardless of what platform is ultimately executing the eventual bundle however due to the nativity of WebGPU to the web ^^ the base of lit-elem is likely to serve the encapsulation which serves as an initializer for the GP2 web library.
Inversion of Control Principles (Running CPU from GPU)
Currently (and correctly) all stateless operations require a stateful launchpad. Any shader or other GPU-based program runs at the behest of a traditional computer's dispatch. A graphics card has no clock by which to time itself except as a count of threads which have been completed within a pass. Therefore, GP2 takes up the task of establishing a configureable back-feed of command from the runner to the dispatcher/orchestrator. Imagine a shader requesting that another shader run without having knowledge of the WebGPU platform but having knowledge of some piece of function not contained in its source and the ability to communicate that some chunk of result is ready to be passed as buffers to that next section of outside code.
The greatest conceptual difference between WebGPU buffers and other GPU memory interfaces is the concept of mapping/unmapping and the lack of ability to even read data off of a mapped object. GP2 has a few strategies for handling transfers based on what the developer ecosystem has so far innovated to handle the deliberate departure from semaphore-managed access which seek to add a conceptual distance between the act of creating and managing low-level patterns such as ring buffering (utilizing several copy destinations in order to prioritize egress of compute results between dispatches without overwrites) in order to emulate common high-level expectations of functional programming. Imagine constantly having to pre-load registers with data instead of passing parameters to a function at call time and suddenly all the conveniences of modern languages become apparent! Let alone lacking a standard approach to achieving "return" statements when CPUs may expect more than "pass" from the asyncronous invocation of an every-frame logic module...when technically "not fail" is still the true implication.
Test-driven capabilities for a linkage protocol
"I need a module that responds with res X to req Y" is a high-level request-style test assertion that GP2 posits will exist as a way for sub-components to extend asks to a motherboard without requiring vendor-specific headers (but allowing 4 them) and that brings me to where I've implemented some assertions so far! Requesting an adapter/device from WebGPU comes with the ability to pass certain flags for either selection of real options visible to the resolver (high or low power for one) and then further if any overrides to the standards defaults on several built-in values can be provided. This is where I am maintaining my "vanilla development" approach by not attempting to rely on any extensions beyond what can be guaranteed for the most mainstream of supported hardware BUT knowing there is a post-captured phase of fail-first testing to determine if potentially more bind groups, RAM etc. can be reserved for a particular GPU application is key to my tools.
The most difficult aspect of building into an in-progress web standard is knowing that one's work may never become part of that standard~ In the case of WebGPU there is only one data type extension which allows for a smaller-than-32 bit variable (f16) and the assumption is that mapping back to a f32 is always an option for the runtime. Unfortunately, if a developer writes a WGSL shader with f16 enabled and attempts to compile that shader in an environment which does not allow f16, the expected behaviour is a failure rather than a fallback. This is why GP2 has the integrated assumption that multiple compile attempts may be expected where the shader logic contains the fallback protocols and there is an intermediary build step for producing the actual WGSL to be run before checking/building any performance indices that may inform specific executions.
One of the most important optional features that cannot be assumed present on a WebGPU device is the timestamp query, which allows for high-resolution timing for sections of code that otherwise can only be measured at the highest level from an external source. With more accurate intervals' measures in GPU time, the actual effort of individual compute steps can be utilized in a planning step once the final source compile is completed for the less granular device options. To give a bad example: if 2 + 2 takes X time and 2 * 2 takes XX time, that cannot be determined until both maths have been tried on the exact combination of hardware and software that is otherwise guaranteed to run the full resultant shader code...So if these otherwise equivalent statements have differing metrics "on my computer, 2 * 2 only takes X time" the best performing fragment can be substituted in-places for the final-final shader code that gets selected and loaded for the aggregate GP2 application.