According to the commentary, the scene where Elton John performs a private performance of "Your Song" to Apu and Manjula on their private garden above the closed Kwik-E-Mart was a reference to Linda and Paul McCartney doing a similar private concert in the ending of Lisa the Vegetarian.
The color of the text in the ending credits is red, instead of yellow, due to the Valentine's Day theme.
The episode title is a spoof of the well known novelty T-shirts with the caption "I'm with stupid".
On a related note, Apu saying "The Bitch is Back" after Elton John snapped at him to lay off the puns to his various works references the namesake of the aforementioned song, where Bernie Taupin's wife at the time, Maxine Feibelman, would use that phrase whenever Elton John was in a bad mood.
There was a King of the Hill episode with the same title. It too is a Valentine's Day episode, though it aired three years later.
When Apu meets Elton John, the latter, after confirming that's his name, sheepishly says "not really" under his breath, referring to how Elton John was initially a stage name and that his birth name was Reginald Dwight. However, the reference was technically inaccurate due to Elton John being his legal name since 1972 (see Goofs section).
When Marge is excited to have received roses from Homer, for a frame her head is misaligned with her necklace so that some space is between some of the beads and her neck, and parts of her neck cover parts of her beads.
When Apu is surprised to learn Elton John's in Springfield, after confirming that's indeed his name, Elton John says "not really" under his breath. In actuality, at the time the episode aired, he had legally changed his name to Elton John, having changed it in 1972.