Empirical Data on Spell Levels?

You know these researchers who published a paper in PLOS ONE are massive nerds. Because the paper determines how hard people think it should be to produce various magical effects. Spell levels by democracy!

D&D spell levels have remained amazingly constant over the different editions of the game -- even when they shouldn't -- so we can see that level in the game is equal parts arbitrary choices and game effect considerations, rather than how much counterfactual power it takes to actually engineer the change.

McCoy J, Ullman T (2019) Judgments of effort for magical violations of intuitive physics. PLoS ONE 14(5): e0217513. https://doi.org/10.1371/journal.pone.021751

Most of these effects are available in the more developed form of AD&D and its followers. To wit:

Conjure: 3 (Monster Summoning I; arguably, conjuring a normal frog is more of a cantrip power)

Cease: 6 (Disintegrate)

Transform: 4 (Polymorph Other)

Split: 8 (Clone)

Stone: 6 (Stone to Flesh)

Invisible: 2 (Invisibility)

Big: 1 (Enlarge)

Teleport: 5 (Teleportation)

Levitate: 2 (Levitate)

Color: 0 (Color cantrip, Unearthed Arcana)

The correlation between these numbers and the intuitive numbers?  A not very impressive r = .43, which means that if you know one of two spells has a higher D&D level, it is only 65% likely it will also have a higher intuitive rank.

This brings us to why spells in D&D from earliest editions to 5th have the level they do, if it's not through some magical model of energy. Yes, it's play balance. Making a frog-sized chunk of most creatures disappear from their anatomy would be more lethal then calling a frog-sized creature into being, even a poisonous one. The same magical physics go into turning a friend and a foe invisible. Merely doubling the mass of a person doesn't have the same delightful possibilities as creating a second, exact duplicate of them. 

AD&D spells, as I've noted before, were not always well designed in the level assigned them. There has been a curious conservatism where spells tend to keep their levels and are more likely to be redesigned to fit their level in power, than to be reassigned level, although some exceptions (like Tasha's Laughter in 5th ed.) can be found. I'm sure this can be backed up by reading Delta's individual "Spells through the Ages" posts, although that scholarly compendium is maddeningly lacking in post tags or a search tool. As an example, Shatter was re-balanced as a damage spell rather than item destruction, which although situational, was quite powerful at the right time. Sleep in fifth edition also had its power curve smoothed out -- not so encounter-ending at early levels, not so useless at late. And Meteor Swarm earns its 9th-level slot by doing five times as much damage as the 3rd-level fireball, as opposed to the AD&D spell whose average damage was not very impressive compared to the average 63 points an 18th-level caster could deal out with a fireball. 

Still, fifth edition has its shares of third-level wizard spells that are nowhere near as useful as the old standards Dispel Magic, Haste, Fireball, and Lightning Bolt, joined by the new wonder-kid on the block, Counterspell (thanks, Wizards). The best that can be said for a Sending, Leomund's Hut, or Phantom Steed is that the wizard memorization economy allows some room for them, and they can be prepared for a special need regardless. Their utility goes up the more the campaign shifts away from toe-to-toe combat and into travel and politics. In that sense, they seem vestigial only as much as difficult travel, communication with allies, and other logistical considerations are brushed over in campaign development.