Many medical researchers have access to large raw data dumps of databases in .csv file format. However, these researchers have no real use for .csv files. They would like to have these files in SQL database format. But, SQL is not something most medical researchers can professionally use. Therefore, they have great need for a bridge between these raw .csv data dumps and a functional SQL database. SQLer is an easy to use tool that converts raw csv data dumps into functional SQL databases. This tool is intended for people without knowledge of SQL and preserves ACID properties. ACID is an acronym for Atomicity, Consistency, Isolation, Durability. This means that the data and the relations between them are properly preserved. This is especially important for researchers in the medical field: they handle large amounts of highly personal, sensitive data. Because many of these researchers may work in secured remote environments, we could not code this project like any other. We practically could not use any external libraries and had to write the code carefully. To ensure functionality on multiple types of (possibly protected) environments, SQLer has to be available in multiple forms. We could also not include any dependencies in the code, such as Python libraries outside of the standard library. To accomplish this, SQLer comes with a command line interface (CLI) and a graphical user interface (GUI). This means that it can be run from a command prompt or terminal. But, for example, it could also be run remotely. The GUI ensures easy usability for users. It offers an easily digestible layout, with a user manual included, to make it as easy as possible for users. SQLer is also usable as a Python package. But, this requires intermediate knowledge of Python, and this is therefore not recommended. Especially not for researchers. SQLer offers users multiple ways to run a tool that converts their .csv raw data dumps to functional SQL databases. This can happen remotely, locally, graphically, and in a terminal. User manuals are provided to help non-technical users of the tool.
Our customer is prof. Marco Spruit from LUMC/LIACS. He had this tool in mind for a long time before we started this project for Software Engineering. For this reason, he was very excited to start the project. And, the longer he waited to start the project, the more researchers he saw struggle with this problem. This was another reason for his excitement. His enthousiasm made our contact enjoyable. We have had good contact with prof. Spruit throughout the course. This was found by us to be very friendly and professional. Prof. Spruit was also helpful: he connected us to a PhD candidate to talk to. He is part of the group who SQLer is intended for. We even had the possibility to demonstrate it during a health club meeting of (PhD) researchers from LUMC and other institutes. We had weekly or twice-a-month meetings. Both for questions and for sprint reviews. It was not always easy to find a good time to meet, as we all have busy schedules. Then again, we could always make it work and prof. Spruit was very flexible.
We can all get along pretty well. Both personally as professionally. This made it noticably easier to meet up. But, especially in the beginning, we struggled with communication. Some work was done twice. We also initially lacked a proper commenting and docstring structure, which led to a lot of trouble understanding each others earlier code. We also did not attend a few of the earlier workgroups. Right up until the very end, some group members did not come to work groups; sometimes without notice. With a proper structure on GitHub.com with issue assignees and milestones we did have some more incentive for the individuals to work on the things they were intended to work on. But, we only did this for two out of the six sprints.