Our project aimed to automate the hyperparameter importance analysis workflow for machine learning models. We combined OpenML with fANOVA and made it all accessible via a webapp.
Users interact with a web interface to select machine learning flows and benchmark suites from OpenML. The app fetches relevant experiment data, applies filters, runs fANOVA in the background, and presents the results through interactive plots and downloadable files.
Our solution simplifies a complex analysis process, making hyperparameter importance analysis more efficient and more accessible. It helps researchers and others identify which parameters matter most, which helps improve model tuning and performance.
Our client was a research team from LIACS, composed of Jan van Rijn and Frank Hutter. The communication was mainly with Jan during our meetings in person every other week. We were also put in touch by Jan via Slack with the team at OpenML to help us with any problems.
We had a scrum master, product owner, meeting note taker and originally also a person for communication with the client. We had meetings with the client every other week and then additionally we had 2 meetings per week with our team. We divided tasks among everyone as equally as possible at the start of the week and then discussed progress etc., at the second meeting of the week. We encountered challenges where some members were more skilled in certain areas which meant they were able to handle tasks better which meant some members could not do much there. We also had issues with communication sometimes with the OpenML team when we needed help with outdated packages and information regarding server setup. As a team we are very proud of our final product and believe it is more than what we originally thought we would have come up with, especially all the different features and its accessibility.