Portable FX Batch Compiler — Automate Shader Compilation for Multiple Targets

Portable FX Batch Compiler: Fast Cross-Platform Shader Building

What it is

• Portable FX Batch Compiler is a command-line tool that compiles shader effect files (FX/HLSL or GLSL variants) in batches, producing platform-specific shader binaries and configurations for multiple graphics backends.

Key features

• Cross-platform: runs on Windows, macOS, and Linux.
• Batch processing: compiles many effect files in one run, useful for CI and build scripts.
• Multiple targets: outputs for Direct3D, Vulkan, Metal, OpenGL, or engine-specific formats.
• Preset profiles: define target platforms, optimization levels, and macro sets.
• Dependency tracking: detects changed sources to recompile only what’s needed.
• Parallel builds: uses multiple CPU cores to speed compilation.
• Error aggregation: collects and formats compiler errors/warnings for easy parsing.
• Customization hooks: pre/post-compile scripts and custom compiler toolchains.

Typical inputs and outputs

• Inputs: .fx, .hlsl, .vert/.frag (GLSL) files; include directories; macro definitions; preset config (JSON/YAML); target lists.
• Outputs: compiled shader binaries (DXBC, DXIL, SPIR-V, Metal library), reflection metadata (JSON), maps for binding locations, logs.

Common use cases

• Game engine asset pipelines — build shaders for multiple platforms automatically.
• Continuous integration — verify shader compilation across targets on pull requests.
• Optimization sweeps — produce builds with different optimization levels for profiling.
• Migration — convert HLSL to SPIR-V or Metal during porting.

Basic workflow (example)

1. Create a config file listing sources, targets, and macros.
2. Run: portable-fx-compiler –config shaders.json –jobs 8
3. Collect outputs into engine asset bundle; inspect logs for errors.

Integration tips

• Commit the config and include paths into repo to ensure reproducible builds.
• Use CI matrix jobs to validate platform-specific outputs if native compilers are required.
• Cache compiled outputs keyed by source hash + config to avoid unnecessary recompiles.

Limitations & gotchas

• Backend compilers (DXC, glslang, metal) still required — the tool orchestrates them but doesn’t replace them.
• Differences in shader semantics across APIs may require conditional code paths or shader variants.
• Reflection formats can vary; ensure your engine’s loader matches the compiler’s metadata output.

If you want, I can:

• generate a sample config file for three targets (Direct3D, Vulkan, Metal),
• provide a CI job example to run the compiler,
• or write a small script to invoke it in parallel.