\TUchapter{SYNCHRONOUS FIRING} \label{ch:Sync-Fire}
\TUsection{Introduction}
One main appeal of attack graphs and compliance graphs are their exhaustiveness. The ability to generate all permutations of attack chains or to generate all
possible ways a system can fall out of compliance is a valuable feature. The disadvantage of this approach is that the generation of the final graph increases
in time, as does the analysis. One other disadvantage is that this exhaustiveness can produce states that are not actually attainable. When a system has assets that
have inseparable features, the generation process forcibly separates features to examine all permutations, since the generation process only modifies one quality at a time. 
One example of an inseparable feature is time. If two different assets are identical and no constraints dictate otherwise, the two assets cannot proceed through times at different rates.

For example, if two cars were manufactured at the same
moment, one of these cars cannot proceed multiple time steps into the future while the other remains at its current time step; each car must step through time at the same rate.
However, the generation of attack graphs and compliance graphs examines the possibilities that one car ages by one time step, while the other car does not, or vice versa. This results in an attack graph 
that can be seen in Figure \ref{fig:non-sync_ex}, which is a partial attack graph showing the separation of the time feature. All shaded states are considered
unattainable, since all of these states comprise of assets that have advanced time at different rates. It is noticeable that not only are the unattainable states themselves a wasteful generation,
but they also lead to the generation of even more unattainable states that will then also be explored. A better procedure for a generation process similar to this example is to have a single state transition that updates
assets with an inseparable feature simultaneously.
\begin{figure}[htp]
    \includegraphics[width=\linewidth]{"./Chapter4_img/non-sync_ex.drawio.png"}
    \vspace{.2truein} \centerline{}
        \caption{A network without Synchronous Firing generating infeasible states}
        \label{fig:non-sync_ex}
\end{figure}

Post-processing is one option at removing the unattainable states. This process would simplify and reduce the time taken for the analysis process, but the generation process would still suffer
from generating and exploring the unattainable states. Instead, a new feature called synchronous firing can be used to prevent the generation of these states. 
The goal of the synchronous firing feature is to prevent the generation of unattainable states, while also not incurring a greater computational cost. This Chapter will discuss the development 
of this feature, discuss its references and inspiration in literature, and examine the results when using this feature in applicable networks.

\TUsubsection{Related Synchronous Firing Work} \label{sec:sync-lit}
A form of synchronous firing is discussed by the author of \cite{louthan_hybrid_2011}, where it is described as grouped exploits. The functionality discussed by the author is similar: firing
an exploit should be performed on all possible assets simultaneously. This was also described as synchronizing multiple exploits. The methodology is similar to the one implemented in this 
work, but there are notable differences. The first, is that the work performed by the author of \cite{louthan_hybrid_2011} utilizes global features with group features. Using the 
simultaneous exploit firing necessitated a separation of global and group features, and grouped exploits could not be performed on exploits that could be applicable to both sets.
A second difference is that there is no consistency checking in the work by the author of \cite{louthan_hybrid_2011}, which could lead to indeterminate behavior or race conditions unless
additional effort was put into encoding exploits to use precondition guards. A third difference is that the work of \cite{louthan_hybrid_2011} could still lead to a separation of features. The
grouped exploit feature would attempt to fire all exploits on all applicable assets simultaneously, but if some assets were not ready or capable to fire, these assets would not proceed with the exploit firing but
the applicable assets would. The last difference is that the work by the author of \cite{louthan_hybrid_2011} was developed in Python, since that was the language of the generator of the tool at the time.
The work by the author of \cite{cook_rage_2018} led to new development of RAGE in C++ for performance enhancements, so the synchronous firing feature in this new work was likewise developed in C++.

\TUsection{Necessary Alterations and Additions}
For the implementation of the synchronous firing feature, there were four primary changes and additions that were necessary. The first is a change in the lexical analyzer, the second involves multiple changes
to PostgreSQL, the third is the implementation of compound operators (as discussed in Section \ref{sec:compops}), and lastly is a change in the graph generation process. This Section will 
consist of subsections discussing the development of these four alterations.

\TUsubsection{GNU Bison and Flex}
The work conducted by the author of \cite{cook_rage_2018} included the introduction of GNU Bison and GNU Flex into RAGE. The introduction of Bison and Flex allows for an easily modifiable grammar to
adjust features, the ability to easily update parsers since Bison and Flex are built into the build system, and increases portability since Flex and Bison generate standard C. For the development of
the synchronous firing feature, a similar approach was taken to that of the work performed by the author of \cite{louthan_hybrid_2011} with the exploit keywords. However, rather than having both global and group
keywords, this work only incorporates the group keyword to prevent a few of the difficulties discussed in Section \ref{sec:sync-lit}. The new ``group" keyword is intended to be used when creating the 
exploit files. The design of exploits in the exploit file is developed as: \begin{verbatim} <exploit>  ::= <group name> "group" "exploit" <identifier> , (<parameter-list>)= \end{verbatim} where the
``$<$group name$>$" identifier
and ``group" keyword is optional. An example of an exploit not utilizing the group feature is: \begin{verbatim}exploit brake_pads(2015_Toyota_Corolla_LE)=\end{verbatim} and an example of an exploit utilizing the group feature is:
\begin{verbatim}time group exploit advance_month(all_applicable)=\end{verbatim}

To implement the keyword recognition and group name parsing, a few changes were made, where the intention was to detect the usage of the ``group" keyword, and have the lexical analyzer code return to the parser implementation file to
alert of the presence of the ``GROUP" token. The new token is of type string with the name ``GROUP", and it is comprised of a leading ``IDENTIFIER" of type string or integer token, followed by the ``GROUP" token.
This new token also required changes to the processing of the ``exploit" keyword. If the group keyword is not detected, the exploit has a group of name ``null". If the group keyword is detected, then the
leading IDENTIFIER is parsed, and the exploit is assigned to a group with the parsed name. Various auxiliary functions were also adjusted to include (for instance) support for printing the groups of each
exploit. Figure \ref{fig:bison-flex} illustrates the incorporation of this feature into Bison, Flex, and the overall program.

\begin{figure}[htp]
    \includegraphics[width=\linewidth]{"./Chapter4_img/Bison-Flex-v2.png"}
    \vspace{.2truein} \centerline{}
        \caption{Inclusion of Synchronous Firing into GNU Bison, GNU Flex, and the overall program}
        \label{fig:bison-flex}
\end{figure}

\TUsubsection{PostgreSQL}
As seen in Figure \ref{fig:bison-flex}, Bison and Flex feed into the Model Database. With the addition of a new group identifier and the group keyword, minor alterations were needed to ensure compatibility 
with the PostgreSQL database. One adjustment was to alter the exploit table in the SQL schema to include new columns of type ``TEXT". The second adjustment was to update the SQL builder functions. This
included updating the related functions such as exploit creations, exploit parsing, database fetching, and SQL string builders to add additional room for the group identifier.

\TUsubsection{Compound Operators}
While not strictly necessary, compound operators greatly simplify the exploit file creation process. For example, implementing time as a feature into the tool without compound operators would increase its difficulty substantially. For each time 
interval, a separate exploit would need to be created, with time flags to indicate the current
time. If time was increased monthly for a year, 12 different exploits would need to
be created, with flags to ensure that time jumps did not occur. Updating qualities without
compound operators also relied on flags, and clever, but convoluted methods for incrementing
values. Instead, compound operators were implemented, and this addition was discussed in Section \ref{sec:compops}.

\TUsubsection{Graph Generation}
The implementation of synchronous firing in the graph generation process relies on a map to hold the fired status of groups. Previously, each iteration of the applicable exploit vector loop generated a new state. With synchronous firing, all assets should be updating the same state, rather than each independently creating a new state. To implement this, each iteration of the applicable exploit vector checks if the current loop element is in a group and if that group has fired. If the element is in a group, the group has not been fired, and all group members are ready to fire, then all group members will loop through an update process to alter the single converged state. Otherwise, the loop will either continue to the next iteration if group conditions are not met, or will create a single state if it is not in a group. Figure \ref{fig:sync-fire} displays the synchronous fire approach.

\begin{figure}[htp]
    \centering
    \includegraphics[scale=0.5]{"./Chapter4_img/Sync-Fire.png"}
    \vspace{.2truein} \centerline{}
        \caption{Synchronous Firing in the Graph Generation Process}
        \label{fig:sync-fire}
\end{figure}


\TUsection{Experimental Networks and Results} \label{sec:test-platform}
All data was collected on a 13 node cluster, with 12 nodes serving as dedicated compute nodes, and 1 node serving as the login node. Each compute node has a configuration as follows:
\begin{itemize}
\item{OS: CentOS release 6.9}
\item{CPU: Two Intel Xeon E5-2620 v3}
\item{Two Intel Xeon Phi Co-Processors}
\item{One FPGA (Nallatech PCIE-385n A7 Altera Stratix V)}
\item{Memory: 64318MiB}
\end{itemize}
All nodes are connected with a 10Gbps Infiniband interconnect.

\TUsubsection{Experimental Networks} \label{sec:Sync-Test}
The example networks for testing the effectiveness of synchronous firing follow the compliance graph generation approach. These networks analyze two assets, both of which are identical 2006 Toyota Corolla cars with identical qualities. The generation examines both cars at their current states, and proceeds to advance in time by a pre-determined amount, up to a pre-determined limit. Each time increment updates each car by an identical amount of mileage. During the generation process, it is determined if a car is out of compliance either through mileage or time since its last maintenance in accordance with the Toyota Corolla Maintenance Schedule manual.

In addition, the tests employ the use of ``services", where if a car is out of compliance, it will go through a correction process and reset the mileage and time since last service. Each test varies in the number of services used. The 1 Service test only employs one service, and it is dedicated to brake pads. The 2 Service test employs two services, where the first service is dedicated to the brake pads, and the second is for exhaust pipes. This process extends to the 3, 4, 5, and 6 Service tests.
The testing information is as follows:
\begin{itemize}
\item{All tests ran for 12 months, with time steps of 1 month.}
\item{All tests had the same number of compliance checks: brake pads, exhaust pipes, vacuum pumps, AC filters, oil changes, and driveshaft boots.}
\item{There were 12 base exploits, and an additional 6 exploits were individually added in the form of services for each test.}
\item{All tests used the same network model.}
\item{All tests used the same exploit file, with the exception of the ``group" keyword being present in the synchronous firing testing.}
\item{Services must be performed prior to advancing time, if services are applicable.}
\item{Graph visualization was not timed. Only the generation and database operation time was measured.}
\end{itemize}

\TUsubsection{Results} \label{sec:Sync-Results}
Using the testing setup described in Section \ref{sec:Sync-Test} on the platform described at the beginning of Section \ref{sec:test-platform}, results were collected in regards to the effect of synchronous firing on both state space and runtime. These results can be seen in Figures \ref{fig:Sync-RT} and \ref{fig:Sync-State}. The respective tables are seen in Tables \ref{table:NS-Table} and \ref{table:S-Table}. Both figures show a decrease in the number of states and a decrease in the runtime when synchronous firing is utilized. Since synchronous firing prevents the generation of unattainable states, there is no meaningful information loss that occurs in the graphs generated with the synchronous firing feature. Since the resulting number of states was also reduced, there will be increased justification for the synchronous firing approach due to a reduced runtime for the analysis process. Figure \ref{fig:Sync-Spd} displays the speedup obtained when using synchronous firing instead of non-synchronous firing for identical setups. Further elaboration is seen in Section \ref{sec:FW}, but it is hypothesized that the synchronous firing approach will lead to an increased runtime reduction and state space reduction when more assets exist in the network due to the increased number of unattainable state permutations.

\begin{figure} 
    \centering
    \includegraphics[width=\linewidth]{"./Chapter4_img/Sync-Runtime-Bar.png"}
    \includegraphics[width=\linewidth]{"./Chapter4_img/Sync-Runtime.png"}
    \caption{Bar Graph and Line Graph Representations of Synchronous Firing on Runtime}
    \label{fig:Sync-RT}
\end{figure}

\begin{figure}
    \centering
    \includegraphics[width=\linewidth]{"./Chapter4_img/Sync-StateSpace-Bar.png"}
    \includegraphics[width=\linewidth]{"./Chapter4_img/Sync-StateSpace.png"}
    \caption{Bar Graph and Line Graph Representations of Synchronous Firing on State Space}
    \label{fig:Sync-State}
\end{figure}

\begin{figure}[htp]
    \centering
    \includegraphics[width=\linewidth]{"./Chapter4_img/Sync_Speedup.png"}
    \vspace{.2truein} \centerline{}
        \caption{Speedup Obtained When Using Synchronous Firing}
        \label{fig:Sync-Spd}
\end{figure}

\begin{table}[htp]
    \centering
    \begin{tabular}{|c|c|c|}
    \hline
        \multicolumn{3}{|c|}{Non-Synchronous Firing} \\ \hline
        \textbf{Number of Services} & \textbf{Number of States} & \textbf{Runtime (ms)} \\ \hline
                1                   &       37001               & 87366.65 \\ \hline
                2                   &       46361               & 115929.97 \\ \hline
                3                   &       72489               & 184634.34 \\ \hline
                4                   &       93525               & 314470.16 \\ \hline
                5                   &       209944              & 588336.01 \\ \hline
                6                   &       423940              & 1581697.61 \\ \hline
    \end{tabular}
    \caption{Tabled Results for the Non-Synchronous Firing Testing}
    \label{table:NS-Table}
\end{table} 

\begin{table}[htp]
    \centering
    \begin{tabular}{|c|c|c|c|}
    \hline
    \multicolumn{4}{|c|}{Synchronous Firing} \\ \hline
    \textbf{Number of Services} & \textbf{Number of States} & \textbf{Runtime (ms)} & \textbf{Speedup}\\ \hline
    1 & 6277 & 14872.86 & 5.87 \\ \hline
        2 & 11653 & 29251.56 & 3.96 \\ \hline
        3 & 25317 & 66799.18 & 2.76 \\ \hline
        4 & 36949 & 102216.30 & 3.08 \\ \hline
        5 & 83134 & 243612.05 & 2.42 \\ \hline
        6 & 186679 & 581840.76 & 2.72 \\ \hline
    \end{tabular}
    \caption{Tabled Results for the Synchronous Firing Testing}
    \label{table:S-Table}
\end{table} 



