diff --git a/lecture-notes-148x210.tex b/lecture-notes-148x210.tex index 9185e28..8328288 100644 --- a/lecture-notes-148x210.tex +++ b/lecture-notes-148x210.tex @@ -110,6 +110,10 @@ \section{Basics of STARKs} \subfile{lectures/13-stark} +\section{Introduction into the Error-Correcting codes} + +\subfile{lectures/14-ecc} + % --- Contact --- \newpage diff --git a/lectures/14-ecc.tex b/lectures/14-ecc.tex new file mode 100644 index 0000000..10ccdc2 --- /dev/null +++ b/lectures/14-ecc.tex @@ -0,0 +1,513 @@ +\documentclass[../lecture-notes-148x210.tex]{subfiles} + +\begin{document} + +\subsection{Introduction} +\textbf{Error-correcting codes (ECC)} are a group of protocols that wrap the +input data extending it with an excessive information that allows recovery of +the original data even if the received message contains errors. Such methods are +widely used in the network protocols and data storage approaches. Moreover, they +found many applications in the modern cryptographic protocols (for example, +Reed-Solomon ECC has been applied in the FRI protocol, which is widely used in +ZK-STARKs). Additionally, each code allows the determination of what errors +exactly appeared in the received message (sometimes, even if the message can not +be decoded). + +ECC protocols can be divided in two groups: \textbf{block codes} and +\textbf{convolutional codes}. The block codes split information into the blocks +of fixed pre-defined size and apply encoding to each block separately, while the +convolutional codes work over the input data stream of an arbitrary size. Modern +practical block codes allow encoding by polynomial complexity algorithms +depending on the block size. Also, most of classic block codes leverage the +properties of the finite fields. + +In this section we primarily describe the structure of some classic block ECCs, +including Walsh-Hadamard code and Reed-Solomon code. Its aim is to give the +reader an understanding of how they work and the mathematical primitives they +rely on. + +\subsection{Preliminaries} + +First of all let's define the main property of each ECC that directly determines its structure: +\begin{definition} +For the arbitrary binary strings of fixed length $x, y \in \{0, 1\}^n$ we define +an \textbf{absolute Hamming distance} as the number of elements where our binary +strings do not match: $\Delta_H(x,y) = \left|\{i \in [n]: x_i \neq y_i\}\right|$. We also +define the \textbf{normalized Hamming distance} as the absolute Hamming distance +divided by the length of the string: $\Delta(x,y) = \frac{1}{n}\left|\{i \in [n]: x_i +\neq y_i\}\right|$. +\end{definition} + +In the theorems below we will use the normalized Hamming distance +$\Delta(\cdot,\cdot)$ because of the independence from codeword length. + +Next, we can define what exactly a block ECC is. In its essence, this is a +function, operating over some field (over binary field $\mathbb{F}_2$, for +example), encoding information using some algorithm $E$ that differs for each +code. Typically, the input to this encoder function is called a \textbf{word} or +\textbf{message}, while the image of such function $\mathsf{im}(E)$ is what we +call a \textbf{codeword}. Let us define that more formally. +\begin{definition} +For each $\delta \in [0, 1]$ we call the image of the encoding function +$\mathsf{Enc}: \{0, 1\}^n \rightarrow \{0, 1\}^m$ an \textbf{error-correcting +code} with distance $\delta$ if for each two \emph{different} input strings $x +\neq y \in \{0, 1\}^n$ the distance between their images is more or equal to +$\delta$: that is, $\Delta(\mathsf{Enc}(x), \mathsf{Enc}(y)) \geq \delta$. In +other words, the code $\mathcal{C} \subseteq \{0,1\}^m$ is the following space: +\begin{equation*} + \mathcal{C} = \mathsf{im}(\mathsf{Enc}) \equiv \{c \in \{0,1\}^m \; \text{such that} \; c=\mathsf{Enc}(x) \; \text{for some} \; x \in \{0,1\}^n \} +\end{equation*} + +To denote such a code, we use notation $[m,n,\delta]$. +\end{definition} + +\begin{remark} + In fact, the distance $\delta$ in the definition above is a lower bound of the + distance between the codewords. This can be formally written as: + \begin{equation*} + \delta = \min_{x \neq y \in \mathsf{im}(\mathsf{Enc})} \Delta(x, y) + \end{equation*} +\end{remark} + +Therefore, each block ECC that takes an input word of size $n$ and outputs a +codeword of size $m>n$ can theoretically deal with $t=m-n$ errors. A block ECC +also utilizes a relation between its distance and its theoretical correcting +capability: $t < \left\lfloor \frac{\delta-1}{2} \right\rfloor$. This relation +exists because two arbitrary codewords can be decoded if they contain at most +$t$ errors. Therefore, the distance between these codewords must be at least +$2t$ to enable unique decoding (if it is less then two different codewords with +errors can be represented as the same message, so the pre-image cannot be found +correctly). While the ECC distance is $\delta$, it leads to the relation above +(check \Cref{fig:distance}). +\begin{figure}[H] + \centering + \includegraphics[width=0.5\linewidth]{images/lecture_14/circles.png} + \caption{Relation between distance and the number of possible errors} + \label{fig:distance} +\end{figure} + +Error-Correcting Codes are extensively used in zk-STARKs. In particular, +we would need to define the so-called \textit{Reed-Solomon Codes}. But +before delving into them, we start from the fundamental construction --- +\textit{Walsh-Hadamard Code}, and consider the properties of ECC based +on it. + +\subsection{Walsh-Hadamard Code} +The Hadamard code, named after French mathematician Jacques Hadamard, is an +error-correcting block code designed for detecting and correcting errors in +message transmissions over highly noisy or unreliable channels. In 1971, NASA +utilized this code to send images of Mars from the Mariner 9 space probe back to +Earth. This code is also referred to as the Walsh code, Walsh family, or +Walsh–Hadamard code, in honor of American mathematician Joseph Leonard Walsh. + +But first, let us define the inner product over binary string. + +\begin{definition}[Inner product] +For two binary strings $x, y \in \{0, 1\}^n$ the \textbf{inner product} $\langle +\cdot, \cdot \rangle: \{0,1\}^n \times \{0,1\}^n \to \mathbb{F}_2$ is defined +as: +\begin{equation*} + \langle x, y \rangle = \sum_{i \in [n]} x_iy_i \; \text{over} \; \mathbb{F}_2 +\end{equation*} + +Equivalently, $\langle x, y \rangle = \bigoplus_{i \in [n]}(x_i \wedge y_i)$ in boolean arithmetic. + +\end{definition} + +Now, we are ready to define the Walsh-Hadamard code. + +\begin{definition}[Walsh-Hadamard code] +The \textbf{Walsh-Hadamard Code} is defined over an encoder function +$\mathsf{Had}: \{0, 1\}^n \rightarrow \{0, 1\}^{2^n}$ that maps each $x \in +\{0,1\}^n$ into the string $z \in \{0, 1\}^{2^n}$ of size $2^n$ where $z_y = +\langle x, y\rangle$ for every $y \in \{0, 1\}^n$. More concisely, this is +written as: +\begin{equation*} + \mathsf{Had}(x) = \left(\langle x, y \rangle\right)_{y \in \{0, 1\}^n} +\end{equation*} +\end{definition} + +\begin{example}[Encoding and Decoding] +For example, for the input size $n=2$, the message space consists of $4$ +elements: $\{0,1\}^2 = \{(0, 0), (0, 1), (1, 0), (1, 1)\}$. Then, for each input codeword +the Walsh-Hadamard code will be: +\begin{equation*} + \mathsf{Had}(x) = (\langle x, y \rangle)_{y \in \{0,1\}^2} = (\langle x, (0, 0) \rangle, \langle x, (0, 1) \rangle, \langle x, (1, 0) \rangle, \langle x, (1, 1) \rangle) +\end{equation*} +For example, taking $x = (0, 1)$ we have: +\begin{gather*} + \mathsf{Had}(\{0, 1\}) = \begin{bmatrix} + \langle (0, 1), (0, 0) \rangle\\ + \langle (0, 1), (0, 1) \rangle\\ + \langle (0, 1), (1, 0) \rangle\\ + \langle (0, 1), (1, 1) \rangle\\ + \end{bmatrix}= \begin{bmatrix} + 0 \cdot 0 + 1 \cdot 0 \\ + 0 \cdot 0 + 1 \cdot 1 \\ + 0 \cdot 1 + 1 \cdot 0 \\ + 0 \cdot 1 + 1 \cdot 1 \\ + \end{bmatrix} = (0, 1, 0, 1) +\end{gather*} +\end{example} + +Now, we consider the decoding procedure of the Walsh-Hadamard ECC. + +\textbf{Decoding.} Suppose we received the codeword $c \in \{0,1\}^{2^n}$ and we +need to decode it to receive the corresponding word $\hat{x} \in \{0,1\}^n$ such +that $c=\mathsf{Enc}(\hat{x})$. + +We use the following technique: for each possible input words $x \in \{0,1\}^n$ +and word $y \in \{0, 1\}^n$, we calculate $\sigma(x,y) = (-1)^{\langle x, y +\rangle}$ (can be done in advance). We represent our received codeword $c$ as a +set of $2^n$ elements $\{(-1)^{c_i}\}_{i \in [2^n]}$. For each possible word $x +\in \{0, 1\}^n$ we calculate the sum +\begin{equation*} + S(x) = \sum_{y \in [2^n]} (-1)^{c_i}\sigma(x, y) +\end{equation*} +This sum represents the similarity between the word $x$ and the encoded word. If +both words have the same sign at position $s$, the sum increases; otherwise, it +decreases. The decoded word $\hat{x}$ is the word $x \in \{0,1\}^n$ with the +highest sum: that is, $\hat{x} \gets \arg\max_{x \in \{0,1\}^n} S(x)$. + +To define the distance for the Walsh-Hadamard ECC we first need to prove an additional lemma: +\begin{lemma}[Random subsum principle] +The following statement holds: +\begin{equation*} + \Pr\left[\langle u, x \rangle \neq \langle v, x \rangle \;\middle| \; + \begin{matrix} + u \neq v \in \{0,1\}^n, \\ + x \xleftarrow{R} \{0,1\}^n + \end{matrix} \right] = \frac{1}{2} +\end{equation*} +\end{lemma} + +\begin{proof} +Note, that $\langle u, x \rangle \neq \langle v, x \rangle$ iff $\langle u, x +\rangle + \langle v, x \rangle = 1$, or, equivalently, $\langle w, x \rangle = +1$ for $w := u+v$ (inner product is linear). We then rewrite this as $\langle w, +x \rangle = \sum_{i \in [n]: w_i \neq 0} x_i = 1$. Since $u \neq v$ from the +initial assumption, $w = u + v$ is not a zero string. Therefore, since $x$ is a +uniformly random string, the equality $\sum_{i \in [n]: w_i \neq 0} x_i = 1$ +holds with probability $\frac{1}{2}$. So, finally, we conclude that $\langle u, +x \rangle \neq \langle v, x \rangle$ with probability $\frac{1}{2}$. +\end{proof} +\begin{lemma} +The Walsh-Hadamard code $\mathsf{Had}$ is an error-correcting code with distance $\delta = \frac{1}{2}$. +\end{lemma} +\begin{proof} +Taking two codewords $\mathsf{Had}(x_1)$ and $\mathsf{Had}(x_2)$, the distance +between them is equal to the number of such $y \in \{0, 1\}^n$ that $\langle +x_1, y \rangle \neq \langle x_1, y \rangle$. We know that this happens in the +half of the cases, so for a codeword of size $2^n$ the distance will be +$\Delta(\mathsf{Had}(x_1), \mathsf{Had}(x_2)) = \frac{2^{n-1}}{2^n} = +\frac{1}{2}$. +\end{proof} +This code made a significant impact on the coding theory, mathematics, and theoretical computer science. However, it is +impractical for modern applications due to the exponential growth in the size of the codewords. + +\subsection{Reed-Solomon code} +In information and coding theory, Reed–Solomon ECCs are a group of block error-correcting codes +developed by Irving S. Reed and Gustave Solomon in 1960. They are widely used across various +applications, including consumer technologies like MiniDiscs, CDs, DVDs, Blu-ray discs, QR codes, and Data Matrix. They +also play a crucial role in data transmission systems such as DSL and WiMAX, broadcast technologies like satellite +communications, DVB, and ATSC, as well as storage solutions such as RAID6. Essentially, the difference between the variants +Reed-Solomon code lies in the assumptions on the basis of which their encoding and decoding algorithms are determined. + +Firstly, let us start from the definition of the input data that differs from +binary: +\begin{definition} +For some alphabet $\Sigma$ and elements $x,y \in \Sigma^n$, we define the +relative distance $\Delta(x, y) = \frac{1}{n}\left|\{i \in [n]: x_i \neq y_i\}\right|$. +\end{definition} +Now we can describe error-correcting codes for the alphabets that differ from +binary. Also, let's describe one subclass of the block ECCs to which the +Reed-Solomon code belong to. +\begin{definition}[Linear code] +Linear code is a code where the set of codewords forms a linear space $\mathcal{C}$ over +field $\mathbb{F}$: +\begin{itemize} + \item[--] For each two codewords $c_1, c_2 \in \mathcal{C}$, the $c_1 + c_2 \in \mathcal{C}$ is also a codeword. + \item[--] For the codeword $c \in \mathcal{C}$ and constant $\lambda \in + \mathbb{F}$ the $\lambda c \in \mathcal{C}$ is a codeword. +\end{itemize} +\end{definition} + +In the subsequent discussion, we will be interested in the special class of +alphabet: $\Sigma = \mathbb{F}_q$. This way, the considered +\emph{linear error correcting code} $\mathcal{C} \subseteq \mathbb{F}_q^n$ is a +linear subspace of the vector space $\mathbb{F}_q^n$. + +\begin{theorem} +Let the \textbf{weight} $\|c\|_H$ of a codeword $c \in \mathcal{C} \subseteq +\Sigma^n$ be the number of positions where its value is non-zero: $\|c\|_H = +\left|\{i \in [n]: c_i \neq 0\}\right|$. + +Then, the distance $\delta$ of the linear error-correcting code is equal to the +minimum possible weight of non-zero codeword: $\delta = \min_{c \in +\mathcal{C}}\|c\|_H$. +\end{theorem} +\begin{proof} +Note that for two codewords $c_1, c_2 \in \mathcal{C}$, the $c_1 - c_2$ is also +a codeword from $\mathcal{C}$. Then, +\begin{equation*} + \delta = \min_{c_1 \neq c_2 \in \mathcal{C}} \Delta(c_1, c_2) = \min_{c_1 \neq c_2} \|c_1 - c_2\|_H = \min_{c \neq 0} \|c\|_H +\end{equation*} + +Note, that the Walsh-Hadamard code is also a linear code, so we can prove its +distance using this approach as well. +\end{proof} + +Now, we shift our focus towards one of the core zk-STARK components: the +Reed-Solomon code. This code is a linear error-correcting code that operates +over the certain finite field $\mathbb{F}$ and polynomials $\mathbb{F}[X]$. Note +that one might encounter different definitions of the Reed-Solomon codes. We +first present the definition used in zk-STARKs and then provide a simplified, +more traditional definition. + +\begin{definition}[Reed-Solomon Code] +Let $\Omega \subseteq \mathbb{F}$ be some \emph{evaluation domain} with a rate +parameter $\rho \in (0,1]$ of size $m := |\Omega|$. Formally, the +\textbf{Reed-Solomon Code} $\mathsf{RS}[\mathbb{F},\Omega,\rho]$ is defined as +the space of functions $f: \Omega \to \mathbb{F}$ that are evaluations of +polynomials of degree up to $\rho m$. Put much more simply, the codeword +of the Reed-Solomon code is an evaluation of a polynomial of degree less than +$\rho m$ over the points of the evaluation domain $\Omega$: +\begin{equation*} + \mathsf{RS}[\mathbb{F},\Omega,\rho] = \left\{f(x)\big|_{x \in \Omega}: f \in \mathbb{F}^{(\leq \rho m)}[x]\right\}, +\end{equation*} + +where we abuse the notation to denote $f(x)\big|_{x \in \Omega} = \left(f(\omega)\right)_{\omega \in \Omega}$. +\end{definition} + +The definition is very hard to grasp from the very first glance, so we help you +with the following example. + +\begin{example} + Suppose $\mathbb{F} = \mathbb{F}_7$ and let evaluation domain be $\Omega = + \{1,2,5,6\}$. Then, the Reed-Solomon code $\mathsf{RS}[\mathbb{F}_7, \Omega, + \frac{1}{2}]$ is a set of all evaluations of polynomials of degree not + exceeding $4 \cdot \frac{1}{2}=2$ (that is, quadratic) over the points + $\{1,2,5,6\}$. For example, for $f(x) = (x+1)^2 \in \mathbb{F}^{(\leq 2)}_7$ + the codeword is $(f(1),f(2),f(5),f(6)) = (4, 2, 1, 0) \in + \mathsf{RS}[\mathbb{F}_7, \Omega, \frac{1}{2}]$. +\end{example} + +\begin{remark} + Typically, we assume that $\rho|\Omega|=\rho m$ is an integer, so we omit rounding + operations in the following discussion. +\end{remark} + +As mentioned earlier, the presented definition can be simplified to the more +traditional one as follows: +\begin{definition}[Reed-Solomon Code (simplified)] +For a field $\mathbb{F}$ and numbers $n \leq m < |\mathbb{F}|$, the +\textbf{Reed-Solomon Code} is a code over the encoding function +$\mathsf{Enc}_{\text{RS}}: \mathbb{F}^n \rightarrow \mathbb{F}^m$ that for each +$n$ degree polynomial $p \in \mathbb{F}[x]$ outputs its evaluation over $m$ +points $x_0,\ldots,x_{m-1} \in \mathbb{F}$. +\end{definition} + +We can show the equality to the original definition if we put $\rho := +\frac{n}{m}$ (note that here $\rho$ has an evident meaning: $\rho$ shows how +many times the evaluation domain is larger than the word space) and $\Omega := +(x_0,\dots,x_{m-1}) \subset \mathbb{F}$: +\begin{equation*} + \mathsf{RS}\left[\mathbb{F},\Omega\,\textcolor{gray}{\left(=\hspace{-2px}\mathbb{F}^m\right)}, \rho\,\textcolor{gray}{\left(=\hspace{-2px}\frac{n}{m}\right)}\right] = \left\{(p(x_0),\dots,p(x_{m-1})): p \in \mathbb{F}^{(\leq n)}[x]\right\}, +\end{equation*} + +\begin{remark} +For the future definitions and theorems we will stick to the original +definition. However, keep in mind that a simplified notation can be obtained by +replacing $\Omega = (x_i)_{i \in [m]}$, $|\Omega| = m$, and $\rho |\Omega| = n$. +\end{remark} + +Let us now estimate some parameters of the Reed-Solomon code. + +\begin{lemma} +The Reed-Solomon code has a distance of $1 - \rho$. +\end{lemma} +\begin{proof} +The word is a polynomial of degree up to $\rho m$ so it can have no more then +$\rho m$ roots. The codeword is the evaluation of this polynomial at $m > \rho +m$ distinct points, meaning it must contain at least $m-\rho m =(1-\rho)m$ +non-zero values (since at most $\rho m$ points can be the roots of our word +polynomial). Since the Reed-Solomon code is a linear code, we can use its +property to define its distance as $\frac{(1-\rho)m}{m} = 1-\rho$. +\end{proof} + +\textbf{Decoding.} By following the ``Unisolvence theorem'' we know that to +recover $\rho m$ degree polynomial we need at least $\rho m + 1$ points for +interpolation. Because the Reed-Solomon code performs polynomial evaluation over +$m > \rho m$ points we can still interpolate (or decode) the polynomial even if +some points are corrupted during data transmission. + +\begin{theorem} +Suppose for a given list of pairs $\mathcal{D} = \{(x_i, y_i)\}_{i \in [m]}$ of +elements in $\mathbb{F}$ there exists a unique polynomial $f \in +\mathbb{F}^{(\leq \rho m)}[x]$ such that $f(x_j) = y_j$ for at least $T := +\frac{1+\rho}{2}m$ of them. Then, there exists a \textit{polynomial-time +algorithm} $\mathsf{Dec}$ (in $O(N^{\gamma})$ for some constant $\gamma>1$), +extracting this very polynomial: $\mathsf{Dec}(\mathcal{D}) = f$. +\end{theorem} + +\textbf{Note.} The assumption that at least $T > \frac{1+\rho}{2}m$ points must +be ``interpolatable'' for some $f \in \mathbb{F}^{(\leq \rho m)}[x]$ implies +that $t < \frac{1}{2}\delta m$ for distance $\delta$. Indeed, $t = m - T$ and +since $T>\frac{1+\rho}{2}m$, we obtain the condition $t < m - \frac{1+\rho}{2}m += \frac{1-\rho}{2}m = \frac{1}{2}\delta m$. + +\begin{proof} Let us construct the decoding algorithm $\mathsf{Dec}$, which is + called the \textit{Berlekamp-Welch decoding}. + +\newpage +\begin{tcolorbox}[title=Berlekamp-Welch Decoding, + breakable, + colback=blue!5!white, + colframe=blue!75!black, + colbacktitle=blue!25!white, + coltitle=blue!20!black, + fonttitle=\bfseries, + boxrule=1.25pt, + subtitle style={boxrule=0pt, + colback=blue!20!white, + colupper=blue!75!gray} ] + + \textbf{Assume:} $f(x_j) = y_j$ for at least $T$ of $m$ pairs in + $\mathcal{D}$ with $f \in \mathbb{F}^{(\leq \rho m)}[x]$. + + \tcbsubtitle{$\mathsf{Dec}(\mathsf{RS}[\mathbb{F},\Omega,\rho], \{x_j, + y_j\}_{j \in [m]})$} + + \begin{itemize}[label=\ding{51}] + \item Introduce the \textbf{error polynomial} $e \in \mathbb{F}^{(\leq + \delta m/2)}[x]$ which has roots at the error points. Notice that the + degree of the error polynomial is less than $\delta m/2$, which follows + from the estimate on $t$. + \item Our algorithm is based on the following equation: + \begin{equation*} + h(x_i) = f(x_i)e(x_i), \; \text{for every} \; i \in [m]. + \end{equation*} + Here, $h(x) \in \mathbb{F}^{\left(\leq \frac{1+\rho}{2}m\right)}$ is an + arbitrary polynomial that we will try to find. Note, that its degree + $\deg h = \deg f + \deg e \leq \frac{1+\rho}{2}m$. + \item Since $f(x_i)=y_i$, the equation translates to $f(x_i)=y_ie(x_i)$ + for each $i \in [m]$, where coefficients of $f$ and $e$ are to be found. + Notice that this is a linear equation over coefficients of $f$ and $e$ + with $\frac{1+\rho}{2}m$ unknowns and $m$ equations. Since $m>\frac{1+\rho}{2}m$ + for any $m$, we are guaranteed to find a solution. + \item Finally, we set $f \gets h/e$. + \end{itemize} +\end{tcolorbox} + +It might be unclear why $e \mid h$ (so that $f$ is not a rational function). +Note that $\mu(x) := f(x)-h(x)e(x)$ is zero in $m$ points and has a degree less +than $m$ (namely, $\rho m$). Therefore, if $\mu$ has $m$ roots and has a degree +$\deg \mu < m$, it must be identically zero. This means that $f(x) = h(x)/e(x)$ +over the whole $\Omega$. +\end{proof} + +\subsubsection{Reed-Solomon(255,223) implementation} +The example implementation aims to help reader to understand how the encoding +and decoding look like in practice. The Reed-Solomon code with $m=255, n=223$ +is widely used in communication and storage systems, including QR codes, CDs, +DVDs, and deep-space communication. It operates over $\mathbb{F}_{256}$ (the +finite field of $256$ elements) and encodes data as polynomials evaluated at +different field elements. Following our theorem, the number of errors +is constrained as $t < \frac{m}{2} - \frac{\rho \cdot m + 1}{2} = \frac{m}{2} - \frac{n}{2}$ +which means that $t < 16$. For our implementation we take $t = 15$. + +Let's start from the basic parameters definition: +\begin{lstlisting}[language=Python,numbers=none] +F = GF(256) +R. = PolynomialRing(F) # define x as a GF(256) element +m = 255 # A codeword size +n = 223 # A word size +t = 15 # The number of possible errors +distance = (m - n + 1)/m +\end{lstlisting} +The \verb|distance| then is equal to $\frac{11}{85}$. Let's sample a random message that will be an information +polynomial of degree $n=223$ (this messsage can be obtained by the interpolation of some useful data): +\begin{lstlisting}[language=Python,numbers=none] +word = sum(F.random_element() * x^i for i in range(n)) +\end{lstlisting} +We start encoding from defining the evaluation elements that will be all non-zero elements of $\mathbb{GF}(256)$. +Then, our codeword will be the $m = 255$ evaluations of our information polynomial. +\begin{lstlisting}[language=Python,numbers=none] +# All non zero elements in F. Order(F) - 1 = 255 +f = [f_i for f_i in F if f_i != 0] +assert len(f) == m +codeword = [word(f_i) for f_i in f] +assert len(codeword) == m +\end{lstlisting} +To decode the codeword we should solve the system of $m$ equations $C(x_i) - b_i \cdot E(x_i) = 0$. Note, that this system +is homogeneous, so the trivial solution (all zeros) is always a solution. To find a non-zero solution we impose a +normalization condition, such as setting the leading coefficient of $E(x)$ to $1$. This turns our original equation +into the $C(x_i) - b_i \cdot E(x_i) = b_i \cdot x_i^{\deg(E)}$. +\begin{lstlisting}[language=Python,numbers=none] +deg_C = 237 # deg = n - 1 + t +deg_E = 15 # deg = t + +M = Matrix(F, m, deg_C + deg_E + 1) +for i, (f_i, b_i) in enumerate(zip(f, codeword)): + # Fill in coefficients for C(x) + for j in range(deg_C + 1): + M[i, j] = f_i^j + # Fill in coefficients for E(x), multiplied by -b_i + for j in range(deg_E): + M[i, deg_C + 1 + j] = -b_i * (f_i^j) + + +rhs = vector(F, [b_i * (f_i^deg_E) for f_i, b_i in zip(f, codeword)]) +# Solve the linear system over F +solution = M.solve_right(rhs) +assert len(solution) == deg_C + deg_E + 1 +\end{lstlisting} +Finally, we can recover the original information polynomial $G(x) = \frac{C(x)}{E(x)}$. We also should not forget to +add the normalized leading coefficient of $E(x)$. +\begin{lstlisting}[language=Python,numbers=none] +C = sum(solution[i] * x^i for i in range(deg_C + 1)) +E = sum(solution[deg_C + 1 + i] * x^i for i in range(deg_E)) +E += x ^ deg_E +G = C/E +assert G.numerator() == word +\end{lstlisting} + +To discrover the error correcting properties of the presented code you can add the following snippet after the +codeword calculation. It changes first $t$ elements of codeword to the random $\mathbb{GF}(256)$ elements. If you try +to change more elements then the linear system solving procedure will fail. +\begin{lstlisting}[language=Python,numbers=none] +for i in range(t): + codeword[i] = F.random_element() +\end{lstlisting} + +\subsection{Convolutional codes} +A convolutional code is an ECC that utilizes the following properties: +\begin{enumerate} +\item[--] For each $k$ input bits it generates $n>k$ output bits; +\item[--] The transformation also depends on the $m$ previous bits; +\item[--] The ECC function is linear: for two inputs $x, y$, corresponding ECC's outputs $X, Y$ and scalars $a,b$ the +following holds: +\begin{equation*} + ax + by = aX + bY +\end{equation*} +\end{enumerate} +For example, the \textbf{linear-feedback shift register} (LFSR) ECC for each $k$ input bits provides $n$ output bits +while remembering $m$ previous bits to use during the encoding process. +\begin{definition}[Convolutional rate] + We call a \textbf{code rate} as the ratio between the input and output sizes: +\begin{equation*} + R = \frac{k}{n} +\end{equation*} +\end{definition} +For example a rate $\frac{1}{3}$ means that for each input bit the code will generate $3$ output bits. Usually, LFSR +operates with rates $\frac{1}{2}$ or $\frac{1}{3}$. Each LFSR code defines the output function per each output bit that +defines how it will be calculated. Usually, this function uses \verb|XOR| over the pre-defined positions of the +previously read values to generate the output (initially they are all zeros). So, during each iteration of the LFSR, +it reads the next bit (or $k$ bits) of the input (which is equivalent to shifting the whole sequence to the right, +which allows to add incoming bits from the left), updates the state of the already read bits (utilizing the +first-in-first-out over the queue of size $m$) and calculates $n$ output bits. + +To decode the output, convolutional codes utilize different algorithms (for example the Sequential Decoding or Viterbi +algorithm), including, but not limited to, heuristic and probabilistic algorithms. All of them differ in computational +complexity and correctness of the result. +\end{document} \ No newline at end of file diff --git a/lectures/images/lecture_14/circles.png b/lectures/images/lecture_14/circles.png new file mode 100644 index 0000000..e9f3e39 Binary files /dev/null and b/lectures/images/lecture_14/circles.png differ diff --git a/preface/acknowledgements.pdf b/preface/acknowledgements.pdf new file mode 100644 index 0000000..a7aca89 Binary files /dev/null and b/preface/acknowledgements.pdf differ diff --git a/preface/preface.pdf b/preface/preface.pdf new file mode 100644 index 0000000..f58438e Binary files /dev/null and b/preface/preface.pdf differ diff --git a/zkdl-template.cls b/zkdl-template.cls index 679a4f7..fe1fb20 100644 --- a/zkdl-template.cls +++ b/zkdl-template.cls @@ -234,7 +234,7 @@ \tcolorboxenvironment{theorem}{ breakable, boxrule=0pt, boxsep=0pt, colback={blue!10},left=8pt,right=8pt,enhanced jigsaw, borderline west={2pt}{0pt}{blue},sharp corners,before skip=10pt,after skip=10pt} \tcolorboxenvironment{lemma}{ breakable, boxrule=0pt, boxsep=0pt, colback={Cyan!10},left=8pt,right=8pt,enhanced jigsaw, borderline west={2pt}{0pt}{Cyan},sharp corners,before skip=10pt,after skip=10pt} \tcolorboxenvironment{corollary}{ breakable, boxrule=0pt, boxsep=0pt, colback={violet!10},left=8pt,right=8pt,enhanced jigsaw, borderline west={2pt}{0pt}{violet},sharp corners,before skip=10pt,after skip=10pt} -\tcolorboxenvironment{proof}{ breakable, boxrule=0pt, boxsep=0pt, blanker,borderline west={2pt}{0pt}{CadetBlue!80!white},left=8pt,right=8pt,sharp corners,before skip=10pt,after skip=10pt} +%\tcolorboxenvironment{proof}{ breakable, boxrule=0pt, boxsep=0pt, blanker,borderline west={2pt}{0pt}{CadetBlue!80!white},left=8pt,right=8pt,sharp corners,before skip=10pt,after skip=10pt} \tcolorboxenvironment{remark}{ breakable, boxrule=0pt, boxsep=0pt, blanker,borderline west={2pt}{0pt}{Green},left=8pt,right=8pt,before skip=10pt,after skip=10pt} \tcolorboxenvironment{remarks}{ breakable, boxrule=0pt, boxsep=0pt, blanker,borderline west={2pt}{0pt}{Green},left=8pt,right=8pt,before skip=10pt,after skip=10pt} \tcolorboxenvironment{example}{ breakable, boxrule=0pt, boxsep=0pt, colback={Gray!10},left=8pt,right=8pt,enhanced jigsaw, borderline west={2pt}{0pt}{Gray},sharp corners,before skip=10pt,after skip=10pt}