Originally, adult sports involving a high level of danger and adrenaline that would not appeal to most people but attract a daring few: things like skydiving, bungee jumping, rock climbing, ice climbing, technical mountaineering, hang gliding. It generally did not include sports which were "extreme" in their early years but since became mainstream (e.g., downhill skiing or scuba diving), nor did it include children's and teenager's activities like BMX or skateboarding. Extreme sports originally meant adult and non-mainstream. Later (from the late 1990s on) the meaning changed to become synonymous with non-dangerous kiddie sports and the definition has more to do with age (and with marketing energy drinks, Mountain Dew, baggy pants, and punk and thrash music), much to the chagrin of older folks involved in dangerous sports who now would rather the term would go away.
Skateboarding? That's not extreme. If you want a real extreme sport try hot air ballooning.