A wise old troll wants to make a small hut. Roofing material costs five dollars per square foot and wall mater

A wise old troll wants to make a small hut. Roofing material costs five dollars per square foot and wall materials cost three dollars per square foot. According to ancient troll customs the floor must be square, but the height is not restricted.
(a) Express the cost of the hut in terms of its height h and the length x of the side of the square floor.
(b) If the troll has only 2535 dollars to spend, what is the biggest volume hut he can build?

I already have a. I got 5x^2+12xh for a but I can't figure out b!

For b, what you have to do is maximize

h * x^2

while

5x^2+12xh = 2535

we can solve for h in terms of x, then use calculus to find the local extrema of [h * x^2]

h = (2535 - 5x^2) / (12x)

plug into

h * x^2

[(2535 - 5x^2) / (12x)] * x^2
=
(2535 / 12) * x - (5 / 12) * x^3

we'll call this f(x)

f(x) = (2535 / 12) * x - (5 / 12) * x^3


now we need to solve f'(x) = 0 so that we can find the local extrema.

f'(x) = (2535 / 12) - (5 / 4) * x^2

f'(x) = 0

(2535 / 12) = (5 / 4) * x^2

x^2 = 169

x = + or - 13

-13 is impossible, so x = 13

now we just plug this into f(x) (f(x) is the function for the volume only in terms of x)

(2535 / 12) * 13 - (5 / 12) * 13^3
=
10985 / 6
=
1830.83 cubic feet