For k ∈ ℝ , let the solutions of the equation ( ^ - 1 (xcot( ^ - 1...

Given,

$\cos \left(\sin^{- 1}\left(xcot\left(\tan^{- 1}\left(\cos \left(\sin^{- 1}x\right)\right)\right)\right)\right) = k$

Now simplifying $\cos \left(\sin^{- 1}x\right) = \cos \left(\cos^{- 1}\sqrt{1 - {x}^{2}}\right) = \sqrt{1 - {x}^{2}}$

So, $\cos \left(\sin^{- 1}\left(xcot\left(\tan^{- 1}\left(\cos \left(\sin^{- 1}x\right)\right)\right)\right)\right) = k$

becomes $\cos \left(\sin^{- 1}\left(xcot\left(\tan^{- 1}\sqrt{1 - {x}^{2}}\right)\right)\right) = k$

And now solving $\cot \left(\tan^{- 1}\sqrt{1 - {x}^{2}}\right) = {cotcot}^{- 1}\left(\sqrt{\dfrac{1}{\sqrt{1 - {x}^{2}}}}\right) = \dfrac{1}{\sqrt{1 - {x}^{2}}}$

So, $\cos \left(\sin^{- 1}\left(xcot\left(\tan^{- 1}\sqrt{1 - {x}^{2}}\right)\right)\right) = k$ becomes

$\cos \left(\sin^{- 1}\left(\dfrac{x}{\sqrt{1 - {x}^{2}}}\right)\right) = k$

Now solving $\cos \left(\sin^{- 1}\left(\dfrac{x}{\sqrt{1 - {x}^{2}}}\right)\right) = \dfrac{\sqrt{1 - 2{x}^{2}}}{\sqrt{1 - {x}^{2}}}$

So, $\dfrac{\sqrt{1 - 2{x}^{2}}}{\sqrt{1 - {x}^{2}}} = k$

$⇒ 1 - 2{x}^{2} = {k}^{2}\left(1 - {x}^{2}\right)$

$⇒ \left({k}^{2} - 2\right){x}^{2} = {k}^{2} - 1$

$⇒ {x}^{2} = \dfrac{{k}^{2} - 1}{{k}^{2} - 2}$

So, roots are $\alpha = \sqrt{\dfrac{{k}^{2} - 1}{{k}^{2} - 2}} ⇒ {\alpha}^{2} = \dfrac{{k}^{2} - 1}{{k}^{2} - 2}$

And $\beta = - \sqrt{\dfrac{{k}^{2} - 1}{{k}^{2} - 2}} ⇒ {\beta}^{2} = \dfrac{{k}^{2} - 1}{{k}^{2} - 2}$

Now finding $\dfrac{1}{{\alpha}^{2}} + \dfrac{1}{{\beta}^{2}} = 2$\left(\dfrac{{k}^{2} - 2}{{k}^{2} - 1}\right)$and$\dfrac{\alpha}{\beta} = - 1$</p><p>So, sum of roots of${x}^{2} - bx - 5 = 0$will be$= \dfrac{1}{{\alpha}^{2}} + \dfrac{1}{{\beta}^{2}} + \dfrac{\alpha}{\beta} = b$</p><p>$⇒ \dfrac{2\left({k}^{2} - 2\right)}{{k}^{2} - 1} - 1 = b ⋯ \left(1\right)$</p><p>Product of roots of${x}^{2} - bx - 5 = 0$will be$= \left(\dfrac{1}{{\alpha}^{2}} + \dfrac{1}{{\beta}^{2}}\right)\dfrac{\alpha}{\beta} = - 5$</p><p>$⇒ \dfrac{2\left({k}^{2} - 2\right)}{{k}^{2} - 1}\left(- 1\right) = - 5$</p><p>$⇒ 2{k}^{2} - 4 = 5{k}^{2} - 5$</p><p>$⇒ 3{k}^{2} = 1 ⇒ {k}^{2} = \dfrac{1}{3} ⋯ Putin\left(1\right)$</p><p>$⇒ b = \dfrac{2\left({k}^{2} - 2\right)}{{k}^{2} - 1} - 1 = 5 - 1 = 4$</p><p>$\dfrac{b}{{k}^{2}} = \dfrac{4}{\dfrac{1}{3}} = 12