21 Definition & Properties

Highlights of this Chapter: we prove many foundational theorems about the derivative that one sees in an early calculus course. We see how to take the derivative of scalar multiples, sums, products, quotients and compositions. We also compute - directly from the definition - the derivative of exponential functions. This leads to an important discovery: there is a unique simplest, or natural exponential, whose derivative is itself. This is the origin of $e$ in Analysis.

Finally - on to some calculus! Here we will define the derivative, and study its properties. This may sound daunting at first, remembering back to the days of calculus when it all seemed so new and advanced. But hopefully, after so much exposure to sequences and series during this course, the rigorous notion of a derivative will feel more just like a nice application of what we’ve learned, than a whole new theory.

21.1 Difference Quotients

The derivative is defined to capture the slope of a graph at a point. Elementary algebra tells us we can compute the slope of a line given two points as rise over run, and so we can compute the slope of a secant line of a function between the points $a, t$ as

$\frac{f (t) - f (a)}{t - a}$

The derivative is the limit of this, as $t \to a$ :

Definition 21.1 (The Derivative) Let $f$ be a function defined on an open interval containing $a$ . Then $f$ is differentiable at $a$ if the following limit of difference quotients exists. In this case, we define the limiting value to be the derivative of $f$ at $a$ . $f^{'} (a) = D f (a) = lim_{t \to a} \frac{f (t) - f (a)}{t - a}$

Exercise 21.1 (Equivalent Formulation) Prove that we may alternatively use the following limit definition to calculate the derivative: $f^{'} (a) = lim_{h \to 0} \frac{f (a + h) - f (a)}{h}$

Example 21.1 The function $f (x) = x^{2}$ is differentiable at $x = 2$ .

This is a classic problem from calculus 1, whose argument is already pretty much rigorous! We wish to compute the limit

$lim_{x \to 2} \frac{x^{2} - 4}{x - 2}$

So, we choose an arbitrary sequence $x_{n}$ with $x_{n} \neq 2$ but $x_{n} \to 2$ and compute

$lim \frac{x_{n}^{2} - 4}{x_{n} - 2} = lim \frac{(x_{n} + 2) (x_{n} - 2)}{x_{n} - 2} = lim x_{n} + 2$

Where the arithmetic is justified since $x_{n} \neq 2$ for all $n$ by definition, so everything is defined. But now, as $x_{n} \to 2$ we can just use the limit laws to see

$lim x_{n} + 2 = 2 + 2 = 4$

Since $x_{n}$ was arbitrary, this holds for all such sequences, so the limit exists and equals 4. Because this limit defines the derivative, we have that $f$ is differentiable at 2 and

$f^{'} (2) = 4$

Exercise 21.2 Compute the derivative of $f (x) = x^{3}$ at an arbitrary point $a \in R$ , directly from the definition and show $f^{'} (a) = 3 a^{2}$ .

As defined above, the derivative is a limit $t \to a$ , which depends on values of $t$ both greater than and less than $a$ . But sometimes its useful to have a notion of the derivative that only cares about one sided limits (for instance, when computing the slope at the end of an interval). We give the analogous definition below

Definition 21.2 (One Sided Derivatives) Let $f$ be a function defined at $a$ ; then its 1-sided derivatives are defined by the following limits, when they exist

$D_{+} f (a) = lim_{t \to a^{+}} \frac{f (t) - f (a)}{t - a}$ $D_{-} f (a) = lim_{t \to a^{-}} \frac{f (t) - f (a)}{t - a}$

This definition, together with our previous work on limits, implies that a function $f$ is differentiable if and only if its two one sided derivatives exist and are equal. This is useful in practice, for instance in showing the non-differentiability of the absolute value:

Exercise 21.3 Show that $f (x) = | x |$ is not differentiable at $x = 0$ .

21.2 Derivative as a Function

So far we have been discussing the derivative at a point as a number; the result of a limiting process. But we can let this point vary, and produce a function taking in $x$ and outputting the derivative at $x$ :

Definition 21.3 (The Function $f^{'}$ ) Let $f$ be a function, and suppose that the derivative of $f$ exists at each point of a set $D \subset R$ . Then we may define a function $f^{'} : D \to R$ by

$f^{'} : x \mapsto f^{'} (x) = lim_{t \to x} \frac{f (t) - f (x)}{t - x}$

If $f^{'}$ is continuous, $f$ is called continuously differentiable on $D$ .

For example, $f (x) = x^{3}$ is continuously differentiable on $R$ since by Exercise 21.2 we see its derivative is the function $x \mapsto 3 x^{2}$ , and this is a polynomial: we proved all polynomials are continuous.

Since the derivative of a function yields another function, we can look at iterating this process to produce higher derivatives

Definition 21.4 (nth Derivatives) Given a differentiable function $f$ , the second derivative $f^{''}$ is defined as the derivative of $f^{'}$ . A function is twice differentiable at $x$ if $lim_{x \to a} \frac{f^{'} (x) - f^{'} (a)}{x - a}$ exists. Continuing inductively, we define the $n^{t h}$ derivative of a function at $a$ as the derivative of the $n - 1^{s t}$ derivative of $f$ at $a$ .

We will use the prime notation for small numbers of derivatives, like $f^{'} (x)$ , $f^{''} (x)$ and $f^{'''} (x)$ . For higher derivatives it is traditional to denote via the number of derivatives in parentheses: $f^{(2)} = f^{''}$ , $f^{(3)} = f^{'''}$ and so on; so $f^{(47)}$ for the 47th derivative of $f$ .

Exercise 21.4 (A Difference Quotient for 2nd Derivative) If $f$ is twice differentiable at $a$ , show that $f^{''} (a) = lim_{h \to 0} \frac{f (a + 2 h) - 2 f (a + h) + f (a)}{h^{2}}$

Find a limit depending only on $f$ (not $f^{'}$ or $f^{''}$ ) which computes the third derivative

Its useful to have a notation for functions which admit $k$ derivatives, we say a function is $C^{k}$ if you can differentiate it $k$ times (but not necessairly $k + 1$ times). And, we call a function smooth if you can differentiate it $n$ times for any $n \in N$ . The set of smooth functions is denoted $C^{\infty}$ .

21.2.1 Continuity

Before jumping in we prove one small oft-useful result often not mentioned in a calculus class, relating differentiability to continuity.

Theorem 21.1 (Differentiable implies Continuous) Let $f$ be differentiable at $a \in R$ . Then $f$ is continuous at $a$ .

Proof. Since $f$ is differentiable at $a$ , we know the limit of the difference quotient is finite $lim_{x \to a} \frac{f (x) - f (a)}{x - a} = f^{'} (a)$ We also know that $lim_{x \to a} (x - a) = 0$ $ So, using the limit theorems we may multiply these together and get what we want. Precisely, let $x_{n} \to a$ be any sequence with $x_{n} \neq a$ for all $n$ . Then we have

$\begin{aligned} 0 & = (0) (f^{'} (a)) \\ = (lim x_{n} - a) (lim \frac{f (x_{n}) - f (a)}{x_{n} - a}) \\ = lim ((x_{n} - a) \frac{f (x_{n}) - f (a)}{x_{n} - a}) \\ = lim (f (x_{n}) - f (a)) \end{aligned}$

Thus $lim (f (x_{n}) - a) = 0$ so by the limit theorems we see $lim f (x_{n}) = a$ . Since $x_{n}$ was arbitrary with $x_{n} \neq a$ this holds for any such sequence, we see that $f$ is continuous at $a$ using the sequence definition.

Remark 21.1. There is a little gap not explicitly spelled out at the end of the proof above, that we should fill in now (to assure ourselves this style of reasoning always works). We just proved that for sequences $x_{n} \neq a$ the property we want holds, but continuity requires this fact for all arbitrary sequences. How do we bridge this gap? Let $y_{n} \to a$ be an arbitrary sequence: then we split into the subsequences $x_{n} \neq a$ and the subsequence of all terms $= a$ . If either of these is finite, we can just truncate the original sequence at a point past which all terms are of one or the other: each of these has $lim f (x_{n}) = f (a)$ so we are done. In the case that both are infinite, we just use that we have separated our sequence into a union of two subsequences, each with the same limit! Thus the overall limit exists.

Thus continuous functions must be differentiable, but what can we say about the derivative itself? If a function is everywhere differentiable must the derivative itself be continuous? In fact not, as the following example shows

Example 21.2 While its hard to imagine a function that is differentiable at every point but not continuously differentiable such things exist. For example $f (x) = {\begin{cases} x^{2} \sin (\frac{1}{x^{2}}) & x \neq 0 \\ 0 & x = 0 \end{cases}$

Its possible to find a formula for $f^{'} (x)$ when $x \neq 0$ , and show that $lim_{x \to 0} f^{'} (x)$ does not exist (we will do this later). However one can also calculate directly the derivative at zero: and find $f^{'} (0) = 0$ . This means $lim_{x \to 0} f^{'} (x) \neq f^{'} (lim_{x \to 0} x)$ as one side does not exist and the other is zero: thus $f^{'}$ is not continuous at $0$ .

Exercise 21.5 For $f (x)$ as above in Example 21.2, calculate $f^{'} (0)$ directly using the limit definition. (Perhaps surprisingly, all you need to know about the sine function here is that it is bounded between $- 1$ and $1$ !)

21.3 Field Operations

Here we prove the ‘derivative laws’ of Calculus I:

21.3.1 Sums and Multiples

Theorem 21.2 (Differentiating Constant Multiples) Let $f$ be a function and $c \in R$ . Then if $f$ is differentiable at a point $a \in R$ so is $c f$ , and $(c f)^{'} (a) = c (f^{'} (a))$

Proof. Let’s use the difference quotient with $a + h_{n}$ to change things up: Let $h_{n} \to 0$ be arbitrary, and we wish to compute the limit $lim \frac{c f (a + h_{n}) - c f (a)}{h_{n}}$ By the limit laws we can pull out the constant $c$ , and the remainder converges to $f^{'} (a)$ , as $f$ is assumed to be differentiable at $a$ .

$= c lim \frac{f (a + h_{n}) - f (a)}{h_{n}} = c f^{'} (a)$

Because this is true for all sequences $h_{n} \to 0$ with $h_{n} \neq 0$ , the limit exists, and equals $c f^{'} (a)$ .

Theorem 21.3 (Differentiating Sums) Let $f, g$ be functions which are both differentiable at a point $a \in R$ . Then $f + g$ is also differentiable at $a$ , and $(f + g)^{'} (a) = f^{'} (a) + g^{'} (a)$

Exercise 21.6 Prove the differentiability rule for sums.

21.3.2 Products and Quotients

Theorem 21.4 (Differentiating Products) Let $f, g$ be functions which are both differentiable at a point $a \in R$ . Then $f g$ is differentiable at $a$ and

$(f g)^{'} (a) = f^{'} (a) g (a) + f (a) g^{'} (a)$

Proof. Let $f, g$ be differentiable at $a \in R$ , and choose an arbitrary sequence $a_{n} \to a$ . Then we wish to compute

$lim \frac{f (a_{n}) g (a_{n}) - f (a) g (a)}{a_{n} - a}$

To the numerator we add $0 = f (a_{n}) g (a) - f (a_{n}) g (a)$ and regroup with algebra:

$= lim \frac{f (a_{n}) g (a_{n}) - f (a_{n}) g (a) + f (a_{n}) g (a) - f (a) g (a)}{a_{n} - a}$ $= lim \frac{f (a_{n}) g (a_{n}) - f (a_{n}) g (a)}{a_{n} - a} + \frac{f (a_{n}) g (a) - f (a) g (a)}{a_{n} - a}$

Using the limit laws, we can take each of these limits individually so long as they exist (which we will show they do). But even more, note that the first term has a common factor of $f (a_{n})$ in the numerator that can be factored out, and the second a common factor of $g (a)$ . Thus, by the limit laws, we see

$= (lim f (a_{n})) (lim \frac{g (a_{n}) - g (a)}{a_{n} - a}) + g (a) (\frac{f (a_{n}) - f (a)}{a_{n} - a})$

Because $f$ is differentiable at $a$ , its continuous at $a$ , and so we know $lim f (a_{n}) = f (a)$ . The other two limits above converge to the derivatives $f^{'} (a)$ and $g^{'} (a)$ respectively. Thus, alltogether we find the resulting limit to be

$f (a) g^{'} (a) + f^{'} (a) g (a)$

As this was the result for an arbitrary sequence $a_{n} \to a$ with $a_{n} \neq a$ , it must be the same for all sequences, meaning the limit exists, and

$(f \cdot g)^{'} (a) = f (a) g^{'} (a) + f^{'} (a) g (a)$

Exercise 21.7 Let $f$ be a function and $a \in R$ be a point such that $f (a) \neq 0$ and $f$ is differentiable at $a$ . Prove that $1 / f$ is also differentiable at $a$ and ${(\frac{1}{f})}^{'} (a) = \frac{- f^{'} (a)}{f (a)^{2}}$

Theorem 21.5 (Differentiating Quotients) Let $f, g$ be a functions which are differentiable at a point $a \in R$ and assume $g (a) \neq 0$ . Then the function $f / g$ is also differentiable at $a$ and ${(\frac{f}{g})}^{'} (a) = \frac{f^{'} (a) g (a) - f (a) g^{'} (a)}{g (a)^{2}}$

Exercise 21.8 Use the Reciprocal Rule and Product Rule to prove the quotient rule.

21.4 Compositions and Inverses

21.4.1 The Chain Rule

Theorem 21.6 (The Chain Rule) If $g (x)$ is differentiable at $a \in R$ and $f (x)$ is differentiable at $g (a)$ then the composition $f \circ g$ is differentiable at $a$ , with $(f \circ g)^{'} (a) = f^{'} (g (a)) g^{'} (a)$

Proof (Wish this Worked!). We are taking the derivative at $a$ , so let $x_{n} \to a$ wtih $x_{n} \neq a$ be arbitrary. Then the limit defining ${[f (g (a))]}^{'}$ is

$lim \frac{f (g (x_{n})) - f (g (a))}{x_{n} - a}$

We multiply the numerator and denominator of this fraction by $ $g (x_{n}) - g (a)$ and regroup:

$\begin{aligned} \frac{f (g (x_{n})) - f (g (a))}{x_{n} - a} & = \frac{f (g (x_{n})) - f (g (a))}{x_{n} - a} \frac{g (x_{n}) - g (a)}{g (x_{n}) - g (a)} \\ = lim \frac{f (g (x_{n})) - f (g (a))}{g (x_{n}) - g (a)} \frac{g (x_{n}) - g (a)}{x_{n} - a} \end{aligned}$

Because $g$ is continuous at $a$ , we know $g (x_{n}) \to a$ , and because $f$ is differentiable at $g (a)$ we recognize the first term here as the limit defining $f^{'}$ at $g (a)$ ! Since the second term is the limit defining the derivative of $g$ , both of these exist by our assumptions, and so by the limit theorems we can compute

$= (lim \frac{f (g (x_{n})) - f (g (a))}{g (x_{n}) - g (a)}) (lim \frac{g (x_{n}) - g (a)}{x_{n} - a})$ $= f^{'} (g (a)) g^{'} (a)$

Unfortunately, this proof fails at one crucial step! Wile we do know that $x_{n} - a \neq 0$ (in the definition of $lim_{x \to a}$ , we only choose sequences $x_{n} \to a$ with $x_{n} \neq a$ ) we do not know that the other denominator $g (x_{n}) - g (a)$ is nonzero.

If this problem could only happen finitely many times it would be no trouble - we could just truncate the beginning of our sequence and rest assured we had not affected the value of the limit. But functions - even differentiable functions - can be pretty wild. The function $x^{2} \sin (1 / x)$ (from Example 21.2) ends up equaling zero infinitely often in any neighborhood of zero! So such things are a real concern.

Happily the fix - while tedious - is straightforward. It’s given below.

Exercise 21.9 We define the auxiliary function $d (y)$ as follows:

$d (y) = {\begin{cases} \frac{f (y) - f (g (a))}{y - g (a)} & y \neq g (a) \\ f^{'} (g (a)) & y = g (a) \end{cases}$

This function equals our problematic difference quotient most of the time, but equals the quantity we want it to be when the denominator is zero.

Prove that $d$ is continuous at $g (c)$ and we may use $d$ in place of the difference quotient in our computation: that for all $x \neq a$ , the following equality holds:

$\frac{f (g (x)) - f (g (a))}{x - a} = d (g (x)) \frac{g (x) - g (a)}{x - a}$

Given this, the original proof is rescued:

Proof. We are taking the derivative at $a$ , so let $x_{n} \to a$ with $x_{n} \neq a$ be arbitrary. Then the limit defining ${[f (g (a))]}^{'}$ is (by the exercise)

$lim \frac{f (g (x_{n})) - f (g (a))}{x_{n} - a} = lim d (g (x_{n})) \frac{g (x_{n}) - g (a)}{x_{n} - a}$

Because $d$ is continuous at $g (a)$ and $g (x_{n}) \to g (a)$ we know $d (g (x_{n})) \to d (g (a)) = f^{'} (g (a))$ . And, as $g$ is differentiable at $a$ we know the limit of the difference quotient exists. Thus, by the limit laws we can separate them and

$= (lim d (g (x_{n}))) (\frac{g (x_{n}) - g (a)}{x_{n} - a}) = f^{'} (g (a)) g^{'} (a)$

21.4.2 Differentiating Inverses

Theorem 21.7 (Differentiating Inverses) Let $f$ be an invertible function and $a \in R$ a point where $f (a) = b$ . Assume $f$ is differentiable at $a$ with $f^{'} (a) \neq 0$ . Then its inverse function $f^{- 1}$ is differentiable at $b$ , and $(f^{- 1})^{'} (b) = \frac{1}{f^{'} (a)}$

One may be tempted to prove this using the chain rule, by the following argument: since $f \circ f^{- 1} (x) = x$ we differentiate to yield $(f \circ f^{- 1} (x))^{'} = 1$ and apply the chain rule to the left hand side, resulting in $f^{'} (f^{- 1} (x)) (f^{- 1})^{'} (x) = 1$

Solving for $(f^{- 1})^{'}$ and plugging in $x = b$ yields the result. However a more careful review shows doesn’t actually do what we think: in applying the chain rule, we’ve implicitly assumed that $f^{- 1}$ is invertible; which is part of what we want to prove! (This proof does go through when we already know $f^{- 1}$ to be differentiable, but we are unfortunately not often already in possession of that knowledge). Below we give a direct proof of the theorem from the limit definition, fixing this oversight:

Proof. We attempt to compute the limit defining the derivative for $f^{- 1}$ : $lim_{y \to b} \frac{f^{- 1} (y) - f^{- 1} (b)}{y - b}$ . To compute such a limit we choose an arbitrary sequence $y_{n} \to y$ with $y_{n} \neq y$ and evaluate $lim \frac{f^{- 1} (y_{n}) - f^{- 1} (b)}{y_{n} - b}$ By definition $b = f (a)$ , and for each $n$ there is a unique $x_{n}$ such that $y_{n} = f (x_{n})$ : making these substitutions yields $lim \frac{f^{- 1} (f (x_{n})) - f^{- 1} (f (a))}{f (x_{n}) - f (a)}$ The composition $f^{- 1} \circ f$ is the identity since they are inverse functions so $f^{- 1} (f (x_{n})) = x_{n}$ and $f^{- 1} (f (a)) = a$ . Making these additional substitutions our limit statement becomes $lim \frac{x_{n} - a}{f (x_{n}) - f (a)}$

By assumption $f$ is differentiable at $a$ and $f^{'} (a) \neq 0$ , so we know that $f^{'} (a) = lim \frac{f (x_{n}) - f (a)}{x_{n} - a}$

The limit we are interested in is the reciprocal of this, and as the limit value is nonzero by assumption, the limit laws imply

$lim \frac{x_{n} - a}{f (x_{n}) - f (a)} = lim \frac{1}{\frac{f (x_{n}) - f (a)}{x_{n} - a}} = \frac{1}{lim \frac{f (x_{n}) - f (a)}{x_{n} - a}} = \frac{1}{f^{'} (a)}$

Since the sequence $y_{n}$ was arbitrary, this argument holds for any such sequence. Thus the limit defining $(f^{- 1})^{'} (b)$ exists, and $(f^{- 1})^{'} (b) = \frac{1}{f^{'} (a)}$ .

Exercise 21.10 Compute the derivative of $y = \sqrt{x}$ using this idea.

21.5 $⧫$ The Power Rule

Perhaps the most memorable fact from Calculus I is the power rule, that $(x^{n})^{'} = n x^{n - 1}$ . In this short section, we prove the power level at various levels of generality, starting with natural number exponents and proceeding to arbitrary real exponents.

Exercise 21.11 (Power Rule: Integer Exponents)

We can use the chain rule, and the the functional equation for roots to differentiate $n$ ^{th}$ roots as well:

Proposition 21.1 If $R (x) = x^{1 / n}$ is the $n^{t h}$ root function, then $R^{'} (x) = \frac{1}{n} x^{\frac{1}{n} - 1}$

Proof. The definition of the $n^{t h}$ root function is that $R (x)^{n} = x$ . We differentiate this equation with the chain rule, using that $n$ is a natural number exponent:

${(R (x)^{n})}^{'} = n R (x)^{n - 1} R^{'} (x)$ The other side was $x$ , whose derivative is $1$ . Thus, $n R (x)^{n - 1} R^{'} (x) = 1$ and, solving for $R^{'}$ yields

$R^{'} (x) = \frac{1}{n R (x)^{n - 1}} = \frac{1}{n (x^{1 / n})^{n - 1}} = \frac{1}{n x^{\frac{n - 1}{n}}}$ $= \frac{1}{n} x^{\frac{1}{n} - 1}$

Exercise 21.12 (Power Rule: Rational Exponents) Run a similar argument to the $n^{t h}$ root case to prove that if $r > 0$ is rational, then $x^{r}$ is differentiable and $(x^{r})^{'} = r x^{r - 1}$ .

When it comes to arbitrary real exponents one can use their definition as limits of rational powers, and work to differentiate such a limit. This is possible but requires an exchange of limits, so needs care. Another method is to use the work we’ve already put into understanding exponentials and logarithms to help us out!

21.6 Problems

The pasting lemma has a differentiable analog, which shows exactly when gluing two pieces (like the absolute value) is differentiable, and when its not.

Exercise 21.13 Let $f, g$ be two continuous and differentiable functions with $a \in R$ a point such that $f (a) = g (a)$ . Prove that the piecewise function $h (x) = {\begin{cases} f (x) & x \leq a \\ g (x) & x > a \end{cases}$ is differentiable at $a$ if and only if $f^{'} (a) = g^{'} (a)$ . (recall we saw such a function is always continuous at $a$ in ?exr-pasting-lemma).

Exercise 21.14 (Differentiable, but The Derivative is Not Continuous) While its hard to imagine a function that is differentiable at every point but not continuously differentiable such things exist. For example $f (x) = {\begin{cases} x^{2} \sin (\frac{1}{x^{2}}) & x \neq 0 \\ 0 & x = 0 \end{cases}$

Assume for the sake of this problem that $\sin (x)$ is a differentiable function on the entire real line, and prove that $f (x)$ is differentiable at every nonzero point, using the product/chain rules.

At $x = 0$ this method fails, but we can compute $f^{'} (0)$ directly using the limit definition. Do this, and show you get zero. (Perhaps surprisingly, all you need to know about the sine function here is that it is bounded between $- 1$ and $1$ !)