The problem statement is:
A common simple digital filter used for smoothing a signal is
$$y(k) = \alpha y(k-1) + (1-\alpha) x(k)\tag{4.59}$$where $\alpha$ is a parameter that determines the response of the filter. Use z-transforms to solve for $y(k)$ as a function of $y(k)$ (assume $y(k < 0) = 0$). What is the amplitude of the frequency response?
Let's start by defining the z-transforms of $y(k)$ and $x(k)$:
$$\begin{cases} Z[y(k)] = \sum_{k=0}^\infty y(k) z^{-k} := Y(z)\\ Z[x(k)] = \sum_{k=0}^\infty x(k) z^{-k} := X(z) \end{cases}\tag{1} $$We have the property $Z[y(k-1)] = \frac{Y(z)}{z}$.
Combining this with equation (4.59) and applying the z-transform to left and right sides, we get:
$$Y(z)\left(1-\frac{\alpha}{z}\right) = (1-\alpha) X(z)\tag{2}$$providing a solution for $Y(z)$:
$$Y(z) = \frac{1-\alpha}{1-\frac{\alpha}{z}} X(z)\tag{2}$$