The name is derived from Desargues' theorem, that describes a configuration of 10 points and 10 lines. The Desargues graph is the bipartite point-line incidence graph of that configuration.
This configuration is found as the configuration of 10 internal points and 10 elliptic lines for a conic in PG(2,5).