The Sixth Lesson on Analysis
Null Functions  in the Theory of Limits of Functions
Last updated 1jul13, previously 19apr11. Added material from 19apr16.

\begin{document}
\maketitle

\section{Introduction}

We do not have time in this edition of the course to delve very deeply
into Chapter 14 "Continuous Functions", and hardly at all into Chapter 15
"Differentiable Functions. Still, it is worthwhile to spend a couple of
lessons on the generalization of null sequences to functions, and show how
this concept simplifies the proof of some significant theorem you will
meet again in a proper analysis course later.

\section{Null Functions}

Recall that a sequence was a function $\mathbb{N} \rightarrow \mathbb{R}$.
Now we are concerned with fucntions $\mathbb{R} \rightarrow \mathbb{R}$.

\textbf{ Definition of a Null function}
$\lim_{h\rightarrow 0} \alpha(h) = 0 \mbox{ means the same as } (\forall \epsilon >0)(\exists \delta > 0)(\forall 0<|h|< \delta)(|\alpha(h)|< \epsilon ).$

Note the similarity of this definition to that of a null sequence. Note also
that the variable $h$ which \textit{vanishes} (=goes to zero) is assumed
not to ever be equal to $0$. This requirement prevents division by zero in
certain cases. Also note, that outside of  physics but in many
applicatios of the calculus, the $h$ is written as $dx$. But we will join
the physicists and avoid this notation. As before, we'll use Greek
letters to denote null functions, and abbreviate
$\lim_{h\rightarrow 0} \alpha(h) = 0$ by $\alpha \rightarrow 0$.

\textbf{ Definition of a Limit}
$\lim_{x\rightarrow a} f(x) = L \mbox{ means the same as } (\exists \alpha \rightarrow 0)( f(a+h) = L + \alpha(h))$

In particular, the \textit{ continuity } of a function $f$ at $x=a$ is
usually defined by writing  $\lim_{x\rightarrow a} f(x)=f(a)$. In view of
our theory of null functions we can write this more practically.

\textbf{ Definition of Continuity}  \\
A function defined in an open interval $(a-e,a-e)=\{ x | |x-a| < e \}$
is said to be continuous at $a$ if there exists a null function defined
for all $h$ with  $0<|h|< e$ for which
$f(a+h) = f(a) + \alpha(h) .$
And we say that a function $f$ is continuous on an entire open interval
$(b,c) =\{x | b < x < c \}$, if it is continuous at every $b< x < c$.

Note that we do need extra letters of the alphabet here. We can equally
well write the coninuity of $f$ on $(b,c)$ by saying
$(\forall b \lt x \lt c)(\exists \mbox{ null } \xi) (f(x+h)=f(x)+\xi(h) ).$

If this is beginning to remind you of Taylor's series in the calculus you
studies earlier, this is no accident. Indeed, we next give a similar
definition of differentiability.

\subsubsection{What is a Neighborhood}
Mathematicians like to abbreviate. Instead of saying that there exists an $e>0$ such that something $P(h)$ is true for all
$0 \lt |x-a| \lt e$, we just say the
propositions $P$ is true in a
\textit{neighborhood of} $x=a$.

\section{Derivatives and Differentiable Functions}

\textbf{ Definition of Derivative } \\
We define a function $f$, which is defined in an open interval contaning
the point $x$, to have a \textit{ derivative} at $x$, if there exists a
real number $m$ (think of "slope" in $y=mx+b$) and null function $\theta$,
so that $f(x+h) = f(x) + mh + \theta(h)h .$

Question 1.

Show that this definition is no different from the classical definition
you learned in the calculus. Hint: Divide through by $h$, rearrange, and
justify that it says $\lim_{h\rightarrow 0}\frac{f(x+h)-f(x)}{h} = m .$

Question 2.

Show that if $f$ has a derivative at $x$ then $f$ is continuous at $x$.
Hint: The difference of $f(x+h)- f(x)$ in the definition is a null function
$\alpha(h) = (m + \theta(h))h$.

From the calculus you remember that the converse of this is not necessarily
true. Continuous functions need not have derivatives. The absolute value
function $y=|x|$ is the usual counterexample for this.

Question 3.

Show why there is no $m$ that satisfies the definition of the
derivative of $f(x)=|x|$ at $x=0.$  Hint: Show that $m$ would have
to have two different values depending on whether $h \lt 0$ and
$0 \lt h$.

We also write $f'(x)$ for the derivative of $f$ at $x$ and dispense with an
extra letter, like $m$, unless we want to emphasize the definition. This
notation also suggests that if $f$ is differentiable at every $x$ in an
open interval, then $f'$ is a function defined on it. Since a function can
have only one value for each argument, we need to realize that \textbf{ if }
a derivative exists, it is unique. We show how versatile null functions are
by proving it this way. If we assume there are two derivatives we have
\begin{eqnarray*}
f(x+h) & = & f(x) + mh + \theta(h)h \mbox{ for some null } \theta \\
f(x+h) & = & f(x) + nh + \psi(h)h \mbox{ for some null } \psi  \\
0 &=&  (m-n)h + (\theta(h) - \psi(h)) h  \\
\mbox{ divide by } h: \,    m-n &=&  \theta(h) - \psi(h) \\
\end{eqnarray*}
Note that here we really use the provision that $h\ne 0$.
Now stare at the last equation. If $m \ne n$, then the LHS is not
$0$. But he RHS becomes smaller than $\epsilon = |m-n|/2$ and we
have a contradiction. Done.

\section{Properties of Null Functions.}
Like the properties of null sequences, somewhere one has to use "epsilonics",
the technique of arguing from the definition. For instance, above we
argued that if $\theta$ is a null function of $h$, so is the function
$\alpha(h) := (m + \theta(h))h.$  This is a special case of the functional
analogue of the theorem that the product of a bounded by a null sequence is
again a null squences.

\subsection{The product rule for null functions}
Let $k(h)$ be a bounded function of $h$. That is, there is a $K>0$ so that
$|k(h)| < K$ for all $h$ in an punctured neighborhood of zero. Then
for any null function $\theta$, their product is null.

\texbf{Proof: } For $\epsilon /K$ there is a $\delta$ so that for all
$0<|h|<\delta$ we have that
$|k(h)\theta(h)|= |k(h)||\theta(h)| < K \epsilon/K = \epsilon.$
Therefore the product is a null function as well.

Question 4.

Prove that the sum of two null functions is again a null function.

\subsection{The composition rule for null functions.}

To show that $\gamma = \beta \circ \alpha$
is still null we need to prove that
$(\forall \epsilon >0)(\exists \delta > 0)(\forall 0<|h|< \delta)(|\gamma(h)|< \epsilon )$
follows from
$(\forall \epsilon >0)(\exists \delta > 0)(\forall 0<|h|< \delta)(|\alpha(h)|< \epsilon )$
and
$(\forall \epsilon >0)(\exists \delta > 0)(\forall 0<|h|< \delta)(|\beta(h)|< \epsilon ).$
We need to manipulate the quantifiers and especially the dummies. We will use subscripts
to indicate which null functions we mean. So given an $\epsilon_\gamma$ (read, "an
epsilon for the composition") use it for $\beta$, i.e. $\epsilon_\beta := \epsilon_\gamma$.
For it there is a $\delta_\beta$ so the $0 \lt |k| \lt \delta_\beta$ implies that
$|\beta(k)| < \epsilon_\beta$. Note how we changed the dummy $h$ to a dummy $k$.

Since we have to use up the hypotheses, we next choose $\epsilon_\alpha := \delta_\beta.$
For this epsilon there is a delta, $\delta_\alpha$ so that
$(\forall 0 < |h| < \delta_\alpha)(|\alpha(h)|< \delta_\beta$.  (I just took two steps in one.)
We conclude that, setting $\delta_\gamma = \delta_\alpha$

\begin{eqnarray*}
|\gamma(h)| &=&|\beta \circ \alpha (h)|  \mbox{ definition of } \gamma  \\
&=&|\beta (\alpha (h))|  \\
& \lt & \epsilon_\beta \mbox{  because } |\alpha(h)| \lt \delta_\gamma \\
& = & \epsilon_\gamma \mbox{  Done ! } \\
\end{eqnarray*}

Note for the purists that we have taken on sall liberty here, we make no claim that
$k := \alpha(h)$ is never zero, which is part of the definition of a null function.
Since this delicacy takes more logic than is good for us right now, I'll skip it.

\subsection{The rules of continuity follow from those of null functions}

For example, not only the sum, difference, product and (some) quotients
of continuous functions are continuous, so is their composition.
We shall prove the composition rule, because it has no analogue among
sequences.

\textbf{ Theorem: } \\
Let $f$ be continuous as $x$ and $g$ be continuous at $f(x)$. Then the
composition $g\circ f$ is continuous at $x$.

\textbf{Proof: } By hypothesis, we write $f(x+h)=f(x)+ \phi(h)$. Next, we
introduce a new variable for convenience and write $y=f(x)$.
The continuity of $g$ at $y$ is written as $g(y+k)=g(y) + \gamma(k)$
for some null function $\gamma$.

Question 5.

Why didn't we just use $h$ again, instead of introducing yet another
variable $k$?

Now, if we decide to set $k=\phi(h)$ and since we have a rule for null
functions that says that their compositions are again null, we
are soon  done. We expand,
\begin{eqnarray*}
g\circ f(x+h) & = &g(f(x+h))=g(f(x) + \phi(h))\\
& = &g(y+k) \\
& = & g(y)+ \gamma(k) \\
& = & g(f(x)) + \gamma(\phi(h)) \\
& = & g\circ f(x) + \gamma \circ \phi(h)) \\
\end{eqnarray*}

\section{Chain Rule}

In calculus you were taught how to use the chain-rule without any indication
of why it should be true. Or better said, which nothing more enlightening
than what Leibniz's ingenious notation for the derivative of $y=f(x)$ as
$\frac{dy}{dx}$ would suggest. You were told to remember that when
$z = g(y)$ then the derivative of $z=h(x)$ where $h(x)=g(f(x))$ was
simply
$\frac{dz}{dx} = \frac{dz}{dy} \frac{dy}{dx}$. This would be obvious if
the $dz, dy, dx$ were numbers. But they aren't. They're infinitesimals.
And if your instructor fumbled with a more rigorouse proof, (s)he probably
fudged it.

With the definition for the derivative above, due to Fr\'echet, in term of
null function, you can \textbf{calculate} the derivative of a composition
without even knowing what it is to begin with.

Now study each step below, where $\phi$ and $\gamma$ are the null functions
known to exist for $f$ and $g$ respectively (hypothesis).
\begin{eqnarray*}
\mbox{ Let } h &=& g\circ f \mbox{ of two differentiable functions } \\
h(x + h) &=& g(f(x+h)) = g(f(x)+ f'(x)h + \phi(h)h) \\
\mbox{ Write } k(h) &=& (f'(x) + \phi(h))h \mbox{ which is null } \\
\mbox{So } h(x+h) &=&  g(f(x)) + g'(f(x))k + \gamma (k)k \\
&=& g(f(x))+ g'(f(x))(f'(x) + \phi(h))h + \gamma(k)(f'(x) + \phi(h))h \\
&=& g(f(x))+ g'(f(x))f'(x)h  + g'(f(x)) \phi(h)h + \gamma(k)(f'(x) + \phi(h))h \\
&=& g(f(x))+ g'(f(x))f'(x)h  + \eta h \\
&=& h(x)+ h'(x)h  + \eta h  \mbox{ which identifies } h'(x) \\

\end{eqnarray*}
Once we prove the lemma that $\eta$ is a null function of $h$ we're done.
Collecting terms, we have
\begin{eqnarray*}
\eta &=& g'(f(x)\phi(h) + \gamma(k(h))f'(x) + \gamma(k(h))\phi(h) \\
&=& const\cdot null + null\circ null \cdot const + null\circ null \cdot null \\
&=& null + null\cdot const + null \cdot null \\
&=& null + null+ null\\
&=& null
\end{eqnarray*}
Now you know why we avoid proving the chain rule in the caclulus.

\subsection{Fudge}
Actually, the above is also fudged a bit. In the original proof you might
have seen in the calculus, in the step
$\frac{\Delta z}{\Delta x}= \frac{\Delta z}{\Delta y} \frac{\Delta y}{\Delta x}$
when the deltas were still real numbers and you couldn't divide by zero, your
instructor had to take two cases. First, for $f'(x) \ne 0$ (Why?), leaving
the case that it is as homework, no doubt.
Well, in Frechet's proof, nothing ever got divided. But there is a subtlety
we sort of skipped over. Can you find it? Here's a chance to win a very fat
bonus in this course!

\subsection{Payoff}
The beauty of Fr\'echet's definition of derivative is in its generalization
to the multivariate calculus (MA241). Because there aren't any denominators,
every product in the formula becomes a scalar, vector, dot, or matrix product
depending on the kinds of functions we're dealing with. All that crazy notation you learned about gradients, curls, and divergences now are unified into a
single concept from linear algebra.

Recall that all of those crazy derivatives involved
partial derivative. So, packaging all the partials of $f(x)$ into the matrix
$f'(x)$ of the appropriate rectangular shape fits into Fr\'echet's formula.
This yields the multivariate interpretation of $dy = f'(x)dx$ as a "small"
displacement of the vector $y$ as the  matrix of partials
$\frac{\partial y}{\partial x}$ applied to the "small" displacement vector
of $x$. This makes mechanics a whole lot easier to work with. But it may
take another century for this improvement to be accepted by the engineering
schools of America.
All you have to put up with is the various kinds of multivariate null functions.
\end{document}