Consider adding other comparison lines

[za158]

Building on #84:

If there is a not nightmarish way to implement this, we might want to include additional comparison lines for some companies. Like when you're looking at a chart for Google, you could see the S&P 500 line, but also a line for Microsoft, Tencent, Meta. Something like that.

[jmelot]

I'm thinking this wouldn't be too difficult to add but what say you @brianlove ? If you think it will be particularly time-consuming maybe we save this for after we've got the MVP together

[brianlove]

@jmelot I've been thinking about how to best handle this, and there are several ways that I've thought of:

1. Each company record includes the cset_ids of companies to compare against it (for example, Company A wants comparisons against B, C, and D). Access the data for B/C/D via the existing company data list
   • Pro: Fairly easy to implement
   • Con: Requires accessing the full data list within DetailView, including loading all those data on every page load for detail view
2. Each company record directly includes the relevant data for any comparison (Company A's entry includes the relevant data for B, C, D; Company B's record includes their comparisons; etc)
   • Pro: Least data use for any specific detail page; easiest to implement
   • Con: Significant duplication of data as company X's data will be included in the record of any company that we compare it against
3. Each company record includes the cset_ids of companies to compare against. The streamlined set of relevant fields for all companies being used for comparisons is included within the overall_data.json file, which is already shared
   • Pro: Minmize data duplication; only data needed for comparisons will be in the new file
   • Con: Inflates the size of overall_data.json; all comparison data still needs to be loaded by the detail pages (company A wants B, C, D, but the data for X, Y, Z will also be downloaded)
4. Each company record includes the cset_ids of companies to compare against. The streamlined set of relevant fields for all companies being used for comparisons is included within a new file (comparison_data.json) (option 3 but in a new file)
   • Pro: Minmize data duplication; only data needed for comparisons will be in the new file
   • Con: All comparison data still needs to be loaded by the detail pages (company A wants B, C, D, but the data for X, Y, Z will also be downloaded)

I think Option 4 is the best route to take, as it minimizes data duplication while being fairly easy to implement. Here's a rough draft of the structure for the comparison_data.json file:

{
  1234: {
    articles: {
      ai_publications: {
        counts: [ 2, 14, 55, ... ],
      },
      ai_pubs_top_conf: {
        counts: [ 0, 2, 1, ... ],
      },
      // ...
    },
    patents: {
      // ...
    },
    other_metrics: {
      // ...
    },
  },
  567: {
    // ...
  },
  // ...
}

This structure could potentially be simplified if we only want comparisons on a small number of charts.

Questions:

• @za158 Will we only want the counts to be in the comparison data, or also rankings? ("Company A ranks #12, while Company X ranks #2, Company Y ranks #13, and Company Z ranks #27"). This affects whether the individual field keys (ai_publications, etc) need to be full objects or can just be the counts array.

[za158]

• @za158 Will we only want the counts to be in the comparison data, or also rankings? ("Company A ranks #12, while Company X ranks #2, Company Y ranks #13, and Company Z ranks #27"). This affects whether the individual field keys (ai_publications, etc) need to be full objects or can just be the counts array.

I don't think we need to compare rankings, especially if it makes a material difference to the engineering.