There's zero reason to remove them, especially when they've already been modeled as part of the house.
And worst case scenario: use your landscaping permit to create a sunken area and plant a line of trees there, then recreate the cliffs all around them. Boom. Apple bush.
That sunken pit will make a fine prison.