If the pair of straight line given by (A x^2+2 H x y+B y^2=0 ), where ( (H^2>A B ) ), forms an equilateral triangle with the line (a x+b y+c=0 ), then ((A+3 B)(3 A+B)= )

Step-by-step solution

According to the given information in the question the angle between lines represented by the equation (A x^2+2 H x y+B y^2=0 ) must be ( 3 ). ( array ll So, & 3 = 2 H^2-A B |A+B| & 3(A+B)^2=4 (H^2-A B ) & 3 A^2+10 A B+3 B^2=4 H^2 & 3 A^2+9 A B+A B+3 B^2=4 H^2 & 3 A(A+3 B)+B(A+3 B)=4 H^2 & (A+3 B)(3 A+B)=4 H^2 array ) Hence, option (a) is correct.