"There is a strange / eerie / eldritch / curious / peculiar / etc. green porcelain urn in the middle of the room."
Leaning on synonyms of "weird" is an understandable residue of the pulp era, although Lovecraft's beloved "queer" is definitively out of play nowadays, at least in that context. But all of these words are crutches for lazy writing. I even suspect that Lovecraft's reliance on them is a strong reason why critics often judge him as a distinctive rather than good prose stylist. Consider this rewrite, which tells more and forces less on the reader:
IMPROVEMENT: "A green porcelain urn is in the middle of the room. It is lopsided, but in a spiralling way that has a certain logic to it, if not symmetry."
Much better. The description creates a vivid image, which the reader instinctively compares with the image of a normal urn. We come by the sense of weirdness through honest imagery rather than a storebought adjective.
These lazy words drive loopholes through a couple of well-known rules in writing. It's widely known by now that a writer should "show, not tell." This is the principle behind the more specific caveat against "mental invasion" text to describe emotional or aesthetic effects in the second person (for example, "A feeling of peace comes over you as you see the unicorn cropping grass in the tranquil glade.")
But "weirdness" words belong to a class of adjectives that essentially lie. They present a subjective reaction of a human being as an objective trait of the world. There are a lot of these words in the emotion lexicon: "disgusting," "creepy," "maddening," "adorable." I think the less a writer uses them, the better. The prose becomes more alive, more muscular, more detached.
Let's see how a sample paragraph from Lovecraft would read with these terms removed or altered. From "The Shadow Over Innsmouth" (which uses 25 "queers" alone, and who knows how many other synonyms...), here's a paragraph which could also stand in for a wordy description of a treasure object, with the disposable weirdness-adjectives in red...
It took no excessive sensitiveness to beauty to make me literally gasp at the strange, unearthly splendour of the alien, opulent phantasy that rested there on a purple velvet cushion. Even now I can hardly describe what I saw, though it was clearly enough a sort of tiara, as the description had said. It was tall in front, and with a very large and curiously irregular periphery, as if designed for a head of almost freakishly elliptical outline. The material seemed to be predominantly gold, though a weird lighter lustrousness hinted at some strange alloy with an equally beautiful and scarcely identifiable metal. Its condition was almost perfect, and one could have spent hours in studying the striking and puzzlingly untraditional designs - some simply geometrical, and some plainly marine - chased or moulded in high relief on its surface with a craftsmanship of incredible skill and grace.In the first-person narration, of course, these terms are more excusable, because they can pass as the honest reaction of the narrator. But it's also striking how little they matter, except in conveying the impression of a highly excitable and creep-prone person through their nearly incantatory use -- and even then I'd say that the "literal gasp" and the "hardly describe" at the beginning are just enough to do that job, while the descriptions of the "elliptical outline" and the "scarcely identifiable metal" continue the sense of weirdness, but through description rather than authorial fiat.