Thursday, June 17, 2021
By:
Switching to the NSF database from the IPEDS database was a difficult decision, and naturally, it gave rise to more decisions. Deciding on how to construct the data tables required lots of experimentation and code review. I realized that I had to carefully choose how to build the data tables so that parsing the data would not take long time computationally. While deciding on the format, I tried many different layouts for the table. I have had to download, upload, re-download and re-upload. After many trials, I think I have found the final format I can follow to make the parsing more efficient! Of course, I have had to reflect all of these changes to the documentation.
It also took me a while to follow and understand the prior code to create the Uberspreadsheet. I found myself lost many times within the many lines of code on my dark VSCode IDE. I think I finally started understanding certain functions. Then, I started planning how I could modify this code to fit my needs when parsing NSF data. Turns out this will be harder than parsing IPEDS data due to the way NSF formulated their “Build a table” property. The grand totals for each race and ethnicity are actually not included in the table and including those would mean losing the grand totals for females and males. Yet again, another decision to make arises!
Another problem was the fact that some of the software and the packages used in creating the program for the Uberspreadsheet in 2019 are not available publicly anymore. Thankfully, I managed to choose a different environment manager and Excel package easily. With this, I will be trying to use dictionaries to my advantage to parse this data efficiently – fingers crossed!
Lastly, we have been receiving messages from more and more institutions about their place on the Compare Your Institution webpage. It seems like we have a lot to update, but that means we have a lot to learn as well. As this “Light meetings week” comes to an end, I wish everyone a good time celebrating/observing Juneteenth!
Zeynep Tuna (she/her/hers)