Solution :
It is given that :
[tex]$\alpha$[/tex] lies in the first quadrant.
And [tex]$\beta$[/tex] lies in the fourth quadrant.
Since, [tex]$\sin \alpha = \frac{5}{13}$[/tex] and [tex]$\cos \beta = \frac{6}{\sqrt{85}}$[/tex] (given)
[tex]$\sin \alpha = \frac{5}{13}$[/tex]
[tex]$\cos \alpha = \sqrt{1-\sin^2 \alpha}$[/tex]
[tex]$\cos \alpha = \frac{12}{13}$[/tex]
Similarly [tex]$\cos \beta = \frac{6}{\sqrt{85}}$[/tex]
[tex]$\sin \beta = \sqrt{1-\cos^2 \beta}$[/tex]
[tex]$\sin \beta = \sqrt{1-\frac{36}{85}}$[/tex]
[tex]$-\frac{7}{\sqrt{85}}$[/tex] (IVth quadrant)
Therefore,
[tex]$\cos (\alpha + \beta) = \cos \alpha \cos \beta - \sin \alpha \sin \beta$[/tex]
[tex]$=\frac{12}{13}\times \frac{6}{\sqrt{85}}-\frac{5}{13}\times \frac{-7}{\sqrt{85}}$[/tex]
[tex]$= \frac{107}{13 \sqrt{85}}$[/tex]
