The castle has a long and varied history. Construction on what was originally a fortress, the only medieval fortress on Åland, began in the late 1300s. When exactly construction began is not known, but the castle is first mentioned in historical sources in 1388. Although the castle was originally located on a small islet, there is now only water on two sides of the building. At the time of the castle’s construction, Åland was an independent administrative district with Kastelholm as its centre.