2.1. Fascia inspired constraint

The human fascia is a network of connective tissues that provides support, protection, and structure to muscles and organs, aiding in movement and shock absorption. It is divided into multiple layers and lines, connecting different muscle groups, and enabling coordinated movements. Myers [Reference Myers14] identified several lines in the body where the fascia connects different muscle groups across various joints. Figure 3 shows the typical fascia lines and their muscle components of the human fascia system.

Figure 3. To introduce the structure and function of human fascia lines, we provide the illustration of (a) the superficial back line (SBL) and (b) the superficial front line (SFL) of the human fascia system [Reference Myers14]. For orange-highlighted parts, (a) the lower SBL from toes to knees, (b) the lower SFL from toes to pelvis. Copyright statement: the figure is sketched referring to the illustration of the book: anatomy trains [Reference Myers14].

When a person moves beyond the natural range of motion, specific fascia lines tense to restrict these movements, causing discomfort or pain. This reaction varies when activating different fascia patterns based on different movements. The tension in fascia lines is influenced by the connected muscles’ activity and condition. The tension within the muscles consequently affects the tension of the associated fascia lines.

The constraint inspired by fascia can become a part of the mechanisms that realize human-like postures and movements, which may benefit in generating posture and movements closer to humans. By exploring the idea of implementing physical constraints inspired by fascia on humanoid robots, there is a possibility that human-like postures and movements could be achieved without the necessity for additional computational costs.

2.2. Actuator network system

To realize such physically connected constraints, we introduce our proposed ANS, a mechanism that forms a connection network by fluid-transmissive tubes composed of cylinders and valves [Reference Ryu, Nakata, Nakamura and Ishiguro17]. ANS changes the movements of cylinders by switching the connection patterns among the actuators.

To perform the switch in ANS, a connection controller is used. When comparing the fast-changing control cycle on a standalone actuator with a feedback control, the connection controller controls actuator groups and does not need to change the control cycle that fast. As a result, the complexity of the control is reduced. Although this method does not allow for precise control of each actuator, the connection enabled simplified control in scenarios where functional synergetic motion is carried out by the interaction of groups of actuators. By physically establishing the constraints among the modular joints, different movement responses of a spinal robot to external forces can be generated without extra computational cost, by switching the connection patterns [Reference Yu, Nakata, Nakamura and Ishiguro18]. Switching the connection patterns of ANS also generated more suitable dynamics for the surrounding environment [Reference Ahmad, Nakata, Nakamura and Ishiguro19, Reference Ahmad, Nakata, Nakamura and Ishiguro20].

Despite our main proposal in this paper being the fascia-inspired constraints, as exploiting ANS indicates the potential of realizing the main proposal, an implementation example of a physical constraint that can be realized by ANS is shown in Section 5.

3.1. Overview of the method

Instead of replicating the original movement with the synergy control only, we consider exploiting the inherent mechanism based on the prior knowledge of the fascia-inspired nonlinear constraints, introduced in Section 2.

Therefore, based on the simplicity of the synergy control method, we aim to introduce constraints that enable the joints to affect mutually, making the robot model generate motions in addition to linear combined motions. In this research, it refers to applying constraints on the multiple DOFs joints in the aspect of rotation angles.

This approach aims to reduce the errors in conventional joint synergy control and ultimately expects to generate movements more reminiscent of human motions. It is divided into 4 steps, as shown in Fig. 4:

Figure 4. The overall flowchart of the proposed method. To show the whole implementation scenario, the approach is divided into 4 steps: 1) Define the base model, 2) reconstruct the joint angles, 3) implement nonlinear movement constraint, 4) optimization.

1) Defining the human model as the base model

Referring to Section 3.2, the human model is defined for the problem statement, which describes the joint rigid structure with a representation of forward kinematics and joint coordinates. It provides a reference for simulating human movements and serves as a benchmark for subsequent modeling steps.

2) Reconstructing the joint angles by extracting and exploiting joint angle synergy

Referring to Section 3.3, the proposed method progresses as an extension to joint synergy; therefore, a synergy extraction method on the defined based model is introduced. We obtained motion sets of reduced dimension joint angles that keep the characteristics of captured human movements. The extraction result is used to reconstruct the joint angles of the human model, as the input of the nonlinear movement constraint implementation.

3) Implementing the nonlinear movement using the proposed model of fascia constraint activation

Referring to Section 3.4, we propose an approach by introducing a nonlinear movement constraint that mimics the mechanism of fascia constraint activation. The human model kinematics is operated through a form of joint angle increment $\boldsymbol{\delta }\textbf{e}$ , which is defined as the angle change that is influenced by the implemented constraints of every control cycle. For the reconstructed input joint angle of each control cycle, to estimate the joint angle increments, the movement of the human model is constrained by different connection patterns. The connection patterns are selected by different constraint activation conditions, according to the calculation result of the fascia constraint model and the threshold presets, as a general determination criterion.

4) Optimization on connection patterns

Referring to Section 3.5, the configuration of connection patterns among the joints is refined through an optimization process. This is achieved by formulating and minimizing error functions that quantify the deviation between the kinematics constrained by our model and those from naive joint synergy.

3.2. Human model

To explain the proposed method, we introduce a 6-joint human model as the base model, to show a whole-body human motion in a 2-D lateral right-hand coordinate frame.

Figure 5. The initial posture and the joint angle definition with the base model. Joint angles are derived from the arc-tangent of the ratio between the y and x coordinates differences of relevant points. For the sake of the figure demonstration, the $J_4$ and $J_5$ in (a) are separated, in fact in the defined human model they coincide, as shown in (b).

Figure 5 shows the definition of the model, including our defined initial posture for subsequent calculation of the fascia constraint model, notated as $\mathbf{q_{ini}}$ . The model consists of 6 active joints, 2 reference joints, and rigid structures.

The active joints fulfill the minimal requirements of performing a whole-body motion of a human. They refer to a joint actuation with the capability to execute controlled rotational movements. In this model, these joints are named as $J_1, J_2, J_3, J_4, J_5, J_6$ (ankle, knee, hip, shoulder, neck, and elbow) as $\textbf{J}_{j}$ $(j=1,2,\ldots,6)$ .

The reference joints are set for calculating $ q_j$ which are adjacent to end effectors. They do not have an actuation mechanism but need to be included in joint angle calculation for biological similarity, as head and wrist joints in this model, named as $R_1, R_2$ (head, wrist).

The $\mathbf{q_{ini}}$ is a starting position where all joint angles are set as $[\pi/2, 0, 0, 0, 0, 0]$ , serving as a reference posture for measuring subsequent movements. The joint angle $ q_j$ is defined to measure the rotation at the active joints, calculated based on the angles between vectors formed by structurally connected joints, the vector relationships of $q_j$ are shown in Fig. 5, and calculated as the following formula:

(1) \begin{equation} q_{\text{j}} = \arctan \left ( \text{v}_{a} \right ) - \arctan \left ( \text{v}_{b}\right ) \end{equation}

For the calculation of $\text{v}$ , refer to Appendix A for the details of the inverse kinematics.

Vise versa, the joint coordinates also can be calculated by the input joint angles with the human model, for error comparison in the optimization. The calculation progress of the direct kinematics is given in Appendix B.

3.3. Joint angle synergy

The purpose of extracting synergy in a multiple-joint robot system is to describe the higher dimensional joint angle space with a low-dimensional synergy space [Reference Zhang, Zhao, Gu and Zhu21], while maximizing the variance of the projected data. For the control simplicity, only dominant synergies are retained.

Principal component analysis (PCA) [Reference Jolliffe and Cadima22], is widely used in extracting the joint angle synergy of robots [Reference Tripathi and Wagatsuma12, Reference Wimböck, Reinecke and Chalon23]. As its efficacy is validated in the literature, in this research, we use PCA to extract the joint angle synergy of the human model.

The inputs of PCA, a dataset of frames of joint angles $\textbf{q}_{\textbf{n}}= \{\textbf{q}_{\textbf{1}}, \textbf{q}_{\textbf{2}}, \textbf{q}_{\textbf{3}}, \textbf{q}_{\textbf{4}}, \textbf{q}_{\textbf{5}}, \textbf{q}_{\textbf{6}}\}$ , where multiple frames $n = 1, \ldots, N$ . The $\textbf{q}_{\textbf{1}}$ to $\textbf{q}_{\textbf{6}}$ is the $n$ frames of joint angles from $J_1$ to $J_6$ .

Implement PCA on the data set, the decomposition result of the $\textbf{q}_{\textbf{n}}$ is

(2) \begin{equation} \mathbf{q_n}=\bar{\mathbf{q}}+\tilde{\mathbf{q}}_{{\textbf{n}}}\mathbf{U} \end{equation}

the dataset joint angle mean value $\bar{\mathbf{q}}$ , eigenvector matrix $\mathbf{U}=\left \{ \mathbf{u_1}, \mathbf{u_2}, \mathbf{u_3}, \mathbf{u_4}, \mathbf{u_5}, \mathbf{u_6}\right \}^T$ , the projection matrix $\tilde{\mathbf{q}}_{\textbf{n}}$ .

An over 80% cumulative contribution rate decides the number of components to retain. Suppose retaining 2 components, obtaining the reconstructed joint angles

(3) \begin{equation} \mathbf{q}_{rec} = \bar{\mathbf{q}} + \mathbf{q}_\boldsymbol{r}\mathbf{V}_{\textbf{s}} \end{equation}

the synergy coordinates $\mathbf{V_s}$

(4) \begin{equation} \mathbf{V}_s = \left \{\mathbf{u}_1, \mathbf{u}_2, \mathbf{0}, \mathbf{0}, \mathbf{0},\mathbf{0}\right \}^T \end{equation}

and the joint inputs

(5) \begin{equation} \mathbf{q}_r = \left \{\tilde{\mathbf{q}}_{\textbf{1}}, \tilde{\mathbf{q}}_{\textbf{2}}, \mathbf{0}, \mathbf{0}, \mathbf{0},\mathbf{0}\right \} \end{equation}

As the $\mathbf{V_s}$ are not time-variant and $\mathbf{q}_\boldsymbol{r}$ is a lower dimensional input signal, in a robot system, such extracted joint angle synergy can reduce the control dimensions of multiple joint inputs, while $\mathbf{q}_{rec} \sim \mathbf{q}_n$ .

The Detailed method of PCA decomposition can be found in Appendix C.

3.4. Fascia-inspired nonlinear constraints

The concept of the proposed method is inspired by the observation that human muscles are grouped and interconnected through fascia. Also, the feature of fascia has been studied by researchers in the robotics field [Reference Liu, Duan, Hitzmann, Xu, Chen, Ikemoto and Hosoda24], by using a robot to study how the foot’s windlass mechanism, comprising plantar fascia, affects jumping, resulting in a simplification of the buliding foot mechanisms of a bipedal robot.

3.4.1. Fascia and fascia constraint model

We suppose that there are multiple fascia lines surrounding the joints of the robot. These “fascia lines” do not exist physically, yet provide morphology criterion by their math models with joint angle inputs. When the robot system tends to perform malformed motion, similar to the human fascia network, the fascia aids and generates responsive constraints or feedback based on the predefined math models. To achieve this, the fascia lines are represented as an extractable line constraint, which is utilized in determining the activation conditions for fascia.

The fascia constraint model is defined as a set of fascia line models $\textbf{L}$ . For a specific fascia line $L \in \textbf{L}$ , connecting designated joints $J \in \textbf{J}_{j}$ referring to the anatomy of human fascia, will increase the tension when the relative joints rotate tending to extract the fascia line, shown in Fig. 6. The extraction direction $\beta$ is defined as $1$ when the placed fascia segment extracts when the joint rotates counterclockwise, $-1$ when the placed fascia segment extracts when the joint rotates clockwise. $\beta$ can only be $+1$ and $-1$ .

Figure 6. The fascia constraint model. The extraction of a segment fascia line is based on its placement and its alignment with the rotation direction. (a) The relationship of rotation direction $q_i - q_{i(0)}$ and the extraction direction $\beta$ . (b) When the joint rotation direction toward $q_{i(0)}$ and extraction direction are the same, the fascia segment extracts. The $\lambda _i$ are assigned first and used to calculate $\kappa$ . (c) For all selected $J_i$ in $J$ , the set of designated joint in a specific fascia line $L$ . (d) The placement of lower SBL and the lower SFL.

A segment fascia tension $\lambda _i$ is defined as the extraction of a segment fascia line of $J_i$ , as a part of the fascia line that surrounds a joint $J_i$ , shown in Fig. 6 (b). Suppose a segment fascia tension $\lambda = 0$ is the initial state of a segment fascia line. For a certain fascia line $L_{linename}$ , the calculation of the fascia tension $\kappa$ of one frame of input angles $\mathbf{q}$ , using the designated joints $J$ , is a cosine value of equally weighted sum up of the designated joints’ segment tension $\{\lambda _i\}$ , For all selected $J_i$ in $J$ , as

(6) \begin{equation} \lambda _i = \begin{cases} \lambda _{0}\cdot \left ({q_i - q_{i(0)}}\right ) & \left ({q_i - q_{i(0)}}\right )\beta \gt 0 \\ \lambda = 0 & \left ({q_i - q_{i(0)}}\right )\beta \leq 0 \end{cases},\forall J_i \in J \end{equation}

(7) \begin{equation} \kappa = \cos \left ( \frac{\sum _{i\in J}\lambda _i J_i}{P} \right ), \forall J_i \in J \end{equation}

where the segment tension is a deviation of input angles of $J_i$ and the corresponding initial angle $q_{i(0)}$ , where $\lambda _{0}=1$ in this research. $P$ is the total number of the designated joints in $J$ . $i$ is the index of the designated joints for $J$ . When there is no extraction on a fascia line, $\kappa =1$ . When there is extraction on a fascia line, $\kappa \lt 1$ .

In this research, we focus on the whole-body motion from the head to the toes. Therefore, we chose the superficial back line (SBL) and the superficial front line (SFL), shown in Fig. 3 as the target of modeling, which has a reciprocal relationship in providing tensile support to lift the rigid which extend backward and forward the gravity line.

Subsequently, we discuss the fascia constraints of the lower SBL and the lower SFL. In the remaining paper, when the fascia line is referred to, we are indicating the fascia constraint model proposed in this research.

3.4.2. Constraint activation conditions for fascia

The constraints of fascia are implemented according to the conditions of the joint angles of every control cycle. The conditions are identified by checking if the tension of a fascia line has surpassed its threshold preset. If surpassed, the correspondent line is treated as "malformed" and labeled as classifying evidence of the current condition. Theoretically, with $C$ fascia lines adapted, $2^C$ conditions are classifiable, within 1 condition of normal, of not malformed condition.

Giving a series of joint angles of a motion set, by using the calculation formula of the fascia tension of a specific line, the time-variant tension can be shown in Fig. 7. For the classified states $S_1, S_2, S_3, S_4$ , the frame sequences that belong to each state are defined as sets $s_1, s_2, s_3, s_4$ , correspondingly, as $s_1, s_2, s_3, s_4 \in \{1, 2, \ldots, N\}$ , while they are independent. The threshold preset is set by human observations, and its configuration is not discussed in this research.

Figure 7. The constraint activation mechanism using fascia tense model. The condition classification is decided by the labels of fascia lines. In this research; 2 fascia lines (lower SBL, lower SFL) are used to constrain the motion frames. Therefore, the motion frames are categorized into 4 conditions: $S_1, S_2, S_3, S_4$ .

Based on the reconstructed joint angle, for every frame (control cycle), the condition of the joint angle is checked in real time. For each time the condition $S_k$ changes, the constraint will switch to the connection pattern $\mathbf{T_k}$ corresponding to the condition, which will be explained in the next subsection. $k$ is the index of the constraint condition pattern, and $K$ is the total number of constraint condition patterns. In this research, we use two lines to activate the constraint condition: lower SBL and lower SFL.

3.4.3. Implementing nonlinear movement constraints on the human model

When the condition of the input motion transits into a condition that the fascia lines are malformed, the nonlinear movement constraint is implemented, to admit a part of joint angles to be transmitted among the joints, pursuing the natural balance of the motion.

A connection pattern is defined for each classified condition, enabling the rotation of one joint to affect the rotation of another one, representing the transition of joint angles. Therefore, in the hardware aspect, the constraint of one joint is a proportional angular displacement from another joint, according to the joint connection pattern configuration at the moment of every control cycle. Figure 8 shows the math representation of the displacement progress.

Figure 8. The math representation of one joint angular displacement from another joint. (a) The physical illustration of $w_{1}\delta{q_2}$ , the displacement using a specific pattern $\mathbf{T_k}$ . (b) The physical meaning of $w_{1}\delta{q_2}$ in different connection patterns $\mathbf{T_K}$ . (c) The physical meaning of $\mathbf{T_k} \cdot \boldsymbol{\delta }\textbf{q}$ , where a specific pattern is illustrated as an example.

In a time-variant system, the whole movement constraint process by initially inputting $\textbf{q}_{\textrm{rec}}$ is, physically, integration progress. In the math representation, for simplicity in a discrete form of the model, we first introduce the increment $\boldsymbol{\delta }\textbf{e}$ , as the cumulative term of angular displacement is influenced by the implemented constraints of every control cycle, a transmission matrix $\mathbf{T_k}$ . Note that $S_k$ and $\mathbf{T_k}$ are bijective, $S_k \to \mathbf{T_k}$ , $\mathbf{T_k} \to S_k$ .

For one $\mathbf{T_k}$ , When the constraint is activated, in every control cycle, the increment $\boldsymbol{\delta }\textbf{e}$ is defined as

(8) \begin{equation} \boldsymbol{\delta }\textbf{e} = T_k \cdot \boldsymbol{\delta }\textbf{q} \end{equation}

where $\boldsymbol{\delta }\textbf{q}$ is the joint angle difference compared to the previous control cycle. And $\mathbf{T_k}$ , $\boldsymbol{\delta }\textbf{q}$ is consists of form

(9) \begin{equation} \begin{bmatrix} 0 & w_{1}^{k} & 0 & 0& 0 & 0\\[5pt] 0 & 0 & w_{2}^{k} & 0& 0 & 0\\[5pt] 0 & 0 & 0 & 0 & w_{3}^{k} & 0\\[5pt] 0 & 0 & 0 & 0 & 0 & 0\\[5pt] 0 & 0 & 0 & 0 & 0 & 0\\[5pt] 0 & 0 & 0 & 0 & 0 & 0 \end{bmatrix} \begin{bmatrix} \delta q_{1}\\[5pt] \delta q_{2}\\[5pt] \delta q_{3}\\[5pt] \delta q_{4}\\[5pt] \delta q_{5}\\[5pt] \delta q_{6} \end{bmatrix} = \begin{bmatrix} w_{1}^{k} \delta q_{2}\\[5pt] w_{2}^{k} \delta q_{3}\\[5pt] w_{3}^{k} \delta q_{5}\\[5pt] 0\\[5pt] 0\\[5pt] 0 \end{bmatrix} \end{equation}

$w$ indicates the transmission proportion of one joint from another joint. In the case of a parameter configuration, as $\delta q_1 = w_{1}^{k} \delta q_{2}$ , $\delta q_2 = w_{2}^{k} \delta q_{3}$ , $\delta q_5 = w_{3}^{k} \delta q_{5}$ , meaning the $J_1$ (ankle) will be constrained by $J_2$ (knee), $J_2$ (knee) will be constrained by $J_3$ (hip), $J_3$ will be constrained by $J_5$ (neck), consistent with the illustration in Fig. 8 (c).

Note the picked-up joints to represent $\mathbf{T_K}$ are different from those in the activation of the fascia constraint model $J$ . The parameter $w$ is what we configure in the next section, meanwhile, the parameter settings and configurations can be varied, although configured as Equation (9) in this research.

To sum up, the constraint method is a time-variant process from the start of a motion depending on the nonlinear switch of the condition patterns as $S_K$ . Consequently, the constrained motion $\textbf{q}_{\textrm{cst}}$ at frame $n$ , $\textbf{q}_{\textrm{cst(n)}}$ is composed of two parts: 1) the reconstructed joint angles of synergy and 2) the nonlinear movement constraint, representing as

(10) \begin{equation} \mathbf{q_{cst(n)}} = \mathbf{q_{rec(n)}} + \sum _{1}^{n}\sum _{k=1}^{K}\boldsymbol{\delta }\textbf{e}\times \mathcal{G}(n) \end{equation}

while

\begin{equation*} \mathcal {G}(n) = \begin {cases} 1 & n \in s_k \\[5pt] 0 & \text {others} \end {cases} \end{equation*}

where $s_k$ is the state set that includes the frame index of the $k$ condition, as mentioned the constraint will adjust the connection pattern that corresponds to the switch of the condition.

3.5. Optimization of the nonlinear movement constraint parameters

We optimize the nonlinear constraint parameters to make the constrained motion become closer to human-performed motion, comparing it with the motion reconstructed by synergy.

To achieve it, the loss function of the whole optimization $\mathcal{L}$ is defined as the comparison of two error functions: 1) between the synergy motion and the original motion $\overline{E_{syn}}$ and 2) between the constrained motion and the original motion $\overline{E_{cst}}$ . This can be represented as

(11) \begin{equation} \mathcal{L} = \overline{E_{cst}} - \overline{E_{syn}} \end{equation}

To calculate them, their general form, the error function $\overline{E}$ , is used to estimate the overall motion performance of all motion sets. Therefore, it is defined as the average of the sum of $E$ . which is the overall error of $N$ frames of all motion sets in Cartesian coordinates by using the direct kinematics human model, inputting the joint angles, as

(12) \begin{equation} \overline{E} = \frac{1}{N}\sum _{i=1}^{N}E \end{equation}

where $N$ is the total frames of all motion sets. $E$ , as the error of one motion frame, is calculated as

(13) \begin{equation} E=\sum _{j=1}^{Z}e_{j(n)} \end{equation}

which $e_{j(n)}$ is the standard square error of the human model for $n$ the frame in the Cartesian coordinates of one joint $J_j$ as

(14) \begin{equation} e_{j(n)}=\sqrt{\left ( y_{j(n)}-\widehat{y_{j(n)}} \right )^2} \end{equation}

$j$ is the joint index, $Z$ is the counts of all joints of human model (including active ones and reference ones), and $n$ is the frame index.

For a series of motion sets, as shown in Fig. 9, $\widehat{\textbf{y}_{\textrm{j(n)}}}$ is the joint coordinates with the original joint angles input, $\textbf{y}_{\textrm{j(n)}}$ is obtained using the human model of direct kinematics with the input $\textbf{q}_{\textrm{(n)}}$ , the joint angle of $n$ th frame. In this research, the joint input of the proposed model $\textbf{q}_{\textrm{cst}}$ and the reconstructed joint angles of synergy model $\textbf{q}_{\textrm{rec}}$ are used.

Figure 9. The error calculation for a series of motion inputs.

The L-BFGS-B [Reference Zhu, Byrd, Lu and Nocedal25] solver is used, as this method optimizes a large number of parameters, using the L-BFGS-B method provides a balance between computation time and accuracy.

To summarize, the loss function, that is, the function to be optimized, is minimized by configuring $w$ in Section 2.4.4, represented as formula

(15) \begin{equation} \begin{aligned} \frac{1}{N}\sum _{n=1}^{N}\Bigg \{ & \sqrt{\sum _{j=1}^{Z}\Bigg [ f\Bigg ( \mathbf{q_{\text{rec(n)}}} + \sum _{1}^{n}\sum _{k=1}^{K}\mathbf{\mathbf{T_k}} \cdot \boldsymbol{\delta }\textbf{q} \times \mathcal{G}(n) \Bigg ) -f(\mathbf{q_{(n)}}) \Bigg ]^{2}} - \sqrt{\sum _{j=1}^{Z}\Bigg [ f\left (\mathbf{q_{\text{rec(n)}}} \right )-f(\mathbf{q_{(n)}}) \Bigg ]^{2}} \Bigg \} \end{aligned} \end{equation}

As $f$ is a continuous function of calculating direct kinematics of the human model, and $\mathbf{T_k}$ is composed of $w^k$ and their derivatives exist. Therefore, the function can be partially differentiated.

The overall optimization step is shown in Algorithm 1.

Algorithm 1 Optimization Using Human and Error Models

4.1. Collection of human movement data via motion capture

We use a motion capture system (Trio tracking system V120, NaturalPoint, Inc. DBA $OptiTrack$ ) to obtain the estimated positions of the joints of a human, by acquiring reflective marker positions when a human subject is performing motions, as shown in Fig. 10. Consequently, we collected 10 kinds of positional motion sets with 11,551 frames of humans using a motion capture system, shown in Table I, the data is the joint angles on the 2-D lateral side of a human, showing a human using his whole body to perform different motions. Each motion is performed 3 to 4 times. Assume one frame is a control cycle.

The marker position using the motion capture system is shown in Fig. 11, in this research, $\mathcal{Q}_1, \mathcal{Q}_2, \mathcal{Q}_4, \mathcal{Q}_{10}$ set as the joint position of $J_1, J_2, J_3, J_6$ of the human model, respectively, $\mathcal{Q}_6$ as the joint position of $J_4, J_5$ , $\mathcal{Q}_8$ as $R_1$ , $\mathcal{Q}_{11}$ as $R_2$ .

Table I. The motion descriptions.

Figure 11. Marker placements on the human body.

4.2. Extraction of joint angle synergy

By adapting the method introduced in Section 2, we extracted the joint synergy from the obtained motion sets. The dataset mean $\bar{\mathbf{q}}=$ [1.32, 0.80, −0.98, −2.50, −0.66, 0.57] (rad), the first 2 synergies (shown in Fig. 12) count for 91.3% cumulative contribution rate. Hence, we choose the first 2 synergies to reconstruct the joint angles as $\textbf{q}_{\textrm{rec}}$ , the reconstruct method is explained in Section 2.

4.3. Optimization of fascia-inspired nonlinear constraints

According to the constraint activation condition calculation method in 2.4.3, for chosen fascia line lower SBL and lower SFL, $\kappa _{LSBL}$ of lower SBL, the designated joint angles $J$ are $J_1, J_2$ : $\kappa _{LSFL}$ of lower SFL, the designated joint angles $J$ are $J_1, J_2, J_3$ . The $\kappa _{LSBL}$ , $\kappa _{LSFL}$ toward frames for M1 to M10 is calculated.

According to the classification method of constraint activation condition in Section 2.4.3, we categorize the motion frames into 4 conditions as mentioned in Fig. 7. Their switching relationship toward time is shown in Fig. 13.

Figure 12. Cumulative contribution rate from the 6 decomposed eigenvectors, 91.3% of the data can be represented by using 2 principal components.

Figure 13. The M1 to M10 motion sets are activated into 4 conditions $S_K$ by frame. The red part demonstrates the scopes of the activated condition $s_k$ .

To achieve a seamless and physically feasible transition when switching motion conditions as in the real world, it is essential to interpolate the state weight correlating with different states. The calculation process is given in Appendix D.

After these configurations, we process the optimization. According to 2.4.4, parameters in $T_1$ , $T_2$ , $T_3$ ( $w_{1}^{1}, w_{2}^{1}, w_{3}^{1}, w_{1}^{2}, w_{2}^{2},w_{3}^{2},w_{1}^{3},w_{2}^{3},w_{3}^{3}$ ) are the optimization variables ( $T_4=0$ as a normal condition). The $T_1$ , $T_2$ , $T_3$ , $T_4$ correspond to 4 conditions. The inputs are the reconstructed joint angles $\mathbf{q_{rec}}$ and constrained joint angles $\mathbf{q_{cst}}$ . Minimize $\mathcal{L} = \overline{E_{cst}} - \overline{E_{syn}}$ of the whole motion set within 200 iterations. The calculation of L-BFGS-B is given in Appendix E.

4.4. Evaluation methodology

We aim to optimize the nonlinear movement constraint parameters to achieve motion that closely resembles human-performed motion.

The optimization involves adjusting parameters $w$ to minimize the loss function defined as the difference between two error functions: the error between the synergy motion and the original motion and the error between the constrained motion and the original motion.

The original motion is a part of collected human movement data, as a one-time performance of each motion collected. The reason for only using a part is to increase the optimization speed in the observation of the homogeneity of multiple times of movements in each kind of motion.

4.5. Results

After 200 iterations, the optimization can get a convergent solution with an average error difference of −0.058, which $\overline{E_{cst}}$ = 0.5197, $\overline{E_{syn}}$ = 0.5781.

From the optimization result, we got the solution of $w$ of $w_{1}^{1} =0.26, w_{2}^{1}=-0.24, w_{3}^{1}=-0.01, w_{1}^{2}=-0.24, w_{2}^{2}=2.35,w_{3}^{2}=-2.02,w_{1}^{3}=-1.09,w_{2}^{3}=-5.78,w_{3}^{3}=3.53$ .

Median errors for each motion set in the proposed and synergy models: Proposed Model/ Synergy Model: M1: 0.362/ 0.377, M2: 0.309/ 0.514, M3: 0.630/ 0.630, M4: 0.543/ 0.595, M5: 0.550/ 0.550, M6: 0.302/ 0.302, M7: 0.256/ 0.263, M8: 0.368/ 0.427, M9: 0.511/ 0.813, M10: 0.403/ 0.738.

We use the optimized parameters to reconstruct the robot motion and visually compare the distribution of error between the proposed model and the synergy model across various motion sets (M1 to M10) with a violin plot shown in Fig. 14.

The human model of M1 motions of the synergy model and the constraint model, which is rebuilt as a series of motion capture clips, is shown in Fig. 15.

Figure 14. The error comparing the synergy model $E_{syn}$ and the proposed model $E_{cst}$ . The distribution is calculated by the $E$ of each frame. The lower the error, the closer to the original motion.

Figure 15. The comparison of video clips of the human model of M1 motions. From left to right: original motion, reconstructed joint synergy model, the proposed model. They are rebuilt with the forward kinematics of the human model. The clips are captured every 30 frames. The connection pattern start switching at f = 161.

Comparing the synergy model, the proposed model is more effective in performing original human motion with a lower median error and lower error distribution.