Reference parameterization
\[
\begin{aligned}
\mathbf{p}_{\text{ref}}(\tau;\alpha) &= \mathbf{C}\,\bm{\beta}(\tau/\alpha) \\
\bm{\beta}(t) &= [1,\,t,\,t^2,\,\ldots,\,t^{N-1}]^\top
\end{aligned}
\]
Closed-loop sensitivity
\(\nabla_\alpha\mathcal{J}_{\text{ATS}}\) is propagated along the real
rollout via a one-step linearisation of the hybrid model. The state
sensitivity \(\mathbf{S}_k \triangleq \mathrm{d}\hat{\mathbf{x}}_k/\mathrm{d}\alpha\),
with \(\mathbf{S}_0 = \mathbf{0}\), evolves recursively:
\[
\mathbf{S}_{k+1} \;=\; \mathbf{A}_k\,\mathbf{S}_k \;+\; \mathbf{B}_k\,\frac{\mathrm{d}\mathbf{u}_k}{\mathrm{d}\alpha}
\]
with the Jacobians frozen at the measured real-world rollout
\((\bar{\mathbf{x}}_k,\bar{\mathbf{u}}_k)\):
\[
\mathbf{A}_k \,\triangleq\, \left.\frac{\partial \mathbf{F}_{\Delta t}}{\partial \mathbf{x}}\right|_{(\bar{\mathbf{x}}_k,\bar{\mathbf{u}}_k)},
\qquad
\mathbf{B}_k \,\triangleq\, \left.\frac{\partial \mathbf{F}_{\Delta t}}{\partial \mathbf{u}}\right|_{(\bar{\mathbf{x}}_k,\bar{\mathbf{u}}_k)}
\]
The action sensitivity is obtained by differentiating the policy
through its observation
\(\mathbf{o}_k(\hat{\mathbf{x}}_k, \mathbf{x}_{\mathrm{ref},k}(\alpha))\):
\[
\frac{\mathrm{d}\mathbf{u}_k}{\mathrm{d}\alpha}
\;=\;
\frac{\partial \pi_\phi}{\partial \mathbf{o}_k}
\!\left(
\frac{\partial \mathbf{o}_k}{\partial \hat{\mathbf{x}}_k}\,\mathbf{S}_k
\;+\;
\frac{\partial \mathbf{o}_k}{\partial \mathbf{x}_{\mathrm{ref},k}}\,
\frac{\partial \mathbf{x}_{\mathrm{ref},k}}{\partial \alpha}
\right)
\]
Chaining \(\mathbf{S}_k\) into the per-step error gradient
\(\partial \mathcal{E}_k/\partial \hat{\mathbf{x}}_k\) closes the
loop: \(\alpha\) is then updated by a single projected gradient
step using the analytical
\(\nabla_\alpha\mathcal{J}_{\text{ATS}}\), with no
finite-difference probing of the physical system.
