That area was "A Natural Fortress"
The unique geography of the Québec City area made it, even before there was any Fortified Wall, "a natural fortress", with high cliffs dominating the St. Lawrence River on both sides of that river (the Québec City north shore and the city of Lévis south shore).
© 2008 Her Majesty the Queen in Right of Canada
In order to better understand where this "natural fortress" came from, we have to look at the geological history of the Québec City area, which dates back from 1,000,000,000 years ago (yes, one "billion" years ago), when our planet had attained approximately its present size and there was only one supercontinent in the middle of only one superocean.
In fact, the previous existence of "a natural fortress" in the Québec City area can better be explained with reference to plate tectonics.
Image Source: en.wikipedia.org
According to the theory of plate tectonics, our planet's crust was broken up into moving plates that rubbed against one another and became modified over time. In the area where Québec City lies today, mountains rose up when the continents collided, and oceans or seas formed when they split apart. These phenomena occurred at an imperceptible rate of 1.6 to 2.4 inches (4 to 6 centimeters) per year, which is about how fast our fingernails grow. Over tens of millions of years and then hundreds of millions of years, however, the position of the continents changed substantially. During such a long time frame, the position of the continents relative to the poles and the equator changed to the point of affecting the climate. We have to remember that even if the continents appear to be stationary when we look at them today because they move so slowly, they have moved over time in the past, are still moving in the present and will move over time in the future. The same logic applies to the different climatic conditions those continents have experienced over time in the past, are still experiencing in the present and will experience over time in the future. In the particular case of the Québec City area, these considerations explain why that area became, gradually and over a very long period of time, "a natural fortress".