SGX3 Hackathon Poster
Abstract
In this project, our team explored data scraping and analysis techniques to extract and evaluate metadata from scientific research papers. As part of the SGX3 Coding Institute, we developed a Python-based workflow that retrieves publication data such as titles, authors, abstracts, citations, and GitHub statistics using APIs like CrossRef and Semantic Scholar. Through hands-on experience with tools including GitHub, Jupyter Notebooks, and Google Colab, we overcame technical challenges related to data formatting, access restrictions, and web scraping limitations. This project reflects our growth in practical coding skills and highlights the importance of reproducibility and open-source contributions in scientific research.
Full text
HACKSTREETBOYS:HACKSTREETBOYS:HACKSTREETBOYS: SGX3 CODING INSTITUTE HACKATHONSGX3 CODING INSTITUTE HACKATHONSGX3 CODING INSTITUTE HACKATHON INTRODUCTIONINTRODUCTIONINTRODUCTION For the SC24 papers, we were able to adjust the parameters to scan the entire website using the HTML code on the website for ISCE to be scraped using our Rubric (you can explain that part), and Gemini is being used to analyze the content scraped from the website. And we got that code on our website, and we were able to fix a problem of Gemini not being able to see the GitHub. There is an issue of the API key running out of request that we hope to find a way to curcumvent in the future. We also want to compile all the scores to make a larger dataset OBJECTIVEOBJECTIVEOBJECTIVE Our objective is to take the scientific research papers provided along with their code and data sets to evaluate their reproducibility along with other metrics that we came up with to give ratings to those papers. Then with those rating create dataset of how many are reproducible or other scientific questions METHODOLOGYMETHODOLOGYMETHODOLOGY We wanted to tackle this duanting task with a lot of caution and space for growth. We created a 4 day plan all spliting the task up, github, building code, presenting, leadership and other skills but our main goals were to; Create the data scraper, provide a dataset to be graphed and get our website up and running. Our analysis focused on extracting metadata from a collection of research articles using data scraping techniques. While the initial setup presented challenges —such as handling inconsistent formats and limited access to certain sources—we refined our approach through iterative testing and the integration of multiple APIs. Ultimately, we developed a functional Python script capable of retrieving key information like abstracts, authors, and citation counts, demonstrating a practical application of data-driven research methods. ANALYSISANALYSISANALYSIS RESULTS/FUTURE WORKRESULTS/FUTURE WORKRESULTS/FUTURE WORK CONCLUSIONCONCLUSIONCONCLUSION Our project focused on evaluating the reproducibility of scientific research papers by analyzing their associated code and datasets. Using a custom rubric, we rated each paper across key metrics and compiled the results into a dataset to explore broader scientific questions. To tackle this challenge, we created a 4-day plan that divided tasks across GitHub setup, code development, data scraping, presentation prep, and leadership roles. We successfully scraped SC24 papers from the ISCE website using adjusted HTML parameters, applied our rubric to evaluate them, and used Gemini to analyze the content. We resolved an issue where Gemini couldn’t detect GitHub links, and all of this was integrated into our working website. AUTHORS Julian Tolbert Yari Pettis Zion Pease Ejay Aguirre Dave Brown . As participants in the SGX3 Coding Institute, our team, The Hackstreet Boys, engaged in a comprehensive four-week program focused on foundational and applied computing skills. Throughout the course, we developed proficiency in Python programming, version control with GitHub, and collaborative coding using Jupyter Notebooks and Google Colab. We also gained practical experience in building and deploying web servers. This poster highlights the key tools, concepts, and projects that shaped our learning journey. TOOLS USEDTOOLS USEDTOOLS USED Chatgpt Gemini API Manus Linux/Git commands Jupiter NoteBook Gemini Github Flask Python Your paragraph text We plan on finding solutions to the issues we were having with the google API keys and their request limits so finding a way to lessen that issue would make the page more accessible. We want to page to display all of the scores data at once into a data set to be used for future reference We want to secure our API keys much better so they aren’t as easily accessible. FUTURE WORKFUTURE WORKFUTURE WORK GITHUB REPOSITORYGITHUB REPOSITORYGITHUB REPOSITORY https://github.com/JulianCT05/SGX3Hackathon