The Scientific Method - Part 1: Hypothesis

Science has been around for centuries and shall continue for many more. The idea that science can be stopped and removed from our lives is unfathomable, simply because we live by its method - The Scientific Method. It is a technique which we use to find answers to our questions, choose which car to buy or which road is optimal to get home quickly after a late dinner. It is a behaviour that we practice but do not put a label on because it comes naturally.

Academically speaking I learned, studied and utilised the method all through my education. During my bachelor degree in physics and mathematics, I continued to learn the tools used for the method and the methodology with which it is applied. I then pursued my doctorate, studying Bayesian statistical techniques in astrophysics and for the past four years, I have been working in Data Science.

So what use does the scientific method have in the industry? In the series of articles titled The Scientific Method, I aim to bridge the gap between data science and business, to communicate scientific terms to non-scientists whilst also encouraging scientists to ask business questions. After all, science thrives on a collaborative effort.

I aim to bridge the gap between data science and business.

Britannica Encyclopedia defines The Scientific Method to be a

mathematical and experimental technique employed in the sciences. More specifically, it is the technique used in the construction and testing of a scientific hypothesis.

It is first and foremost a method used by scientists to test an idea, a hunch or in scientific terms, a hypothesis. However, its vague or rather broad definition shows how flexible the method is to be utilised in different sectors of science. The process is also applicable in the science of data, Data Science, an ever-growing field in the industry. In this article, I will focus on how the hypothesis is built academically but mostly industrial, and always scientifically.

During my Bachelor degree, we had a weekly, or bi-weekly scientific experiment to conduct. The title usually defines the problem and in a section under the title referred to as Aim, we would have a description of the result that we seek or the theory we want to test or confirm. All this sounds very abstract unless you have been in one of the science labs, so let me translate.

Academically this would usually be done by having a research problem, whereby the researcher would have to do extensive literature reading to understand the history of the problem, results already published and persistent limitations encountered throughout. More often, collaborations with other teams are encouraged to share knowledge first-hand. It is the same thing in the industry but I learned that a different vocabulary is used in practice.

Business managers or product owners alike, a.k.a. businesspeople, these individuals know the business and the rhythm by which its heart is beating. They usually work closely with the customers and therefore are exposed to feedback. Businesspersons either receive new ideas that would alleviate some of the struggles that clients face or suggest a product that would generate revenue. Either way, if the problem requires data-driven insight and actions, the businessperson reaches out to the Data Science team and makes a request, a project proposal.

That proposal is the title of the experiment in the analogy of a lab experiment. Based on my personal experience, the title would be catchy but initially very vague.

I can share that "understanding customer behaviour" has been a common question in all the companies I worked for, but the type of behaviour that each business tried to understand was completely different.

So how do we clear out the ambiguity of the question and write a solid description of our Aim?

Humans may think of ourselves (arguably) as the most intelligent species on the planet. However, speaking for myself perhaps, I cannot read people's minds. I do not wish to acquire that talent so instead, I practice the art of asking questions. In other words, I ask the individual putting the proposed project forward a lot of questions to help me construct the hypothesis.

The construction of the hypothesis is paramount to set the tone, requirements and paint the picture of what the project will represent. During this stage, the businessperson is the one who has most of the answers or knows why the experiment is required. On the other hand, the data scientist may have a vague understanding but for the sake of this article, a data scientist who has limited experience in that particular industry would not know, on the spot, what the person's intentions are.

There it begins, questions ready to be exchanged which boil down to the famous three:

• Why

• What

• How

I enjoy doing my work, I love it really, but I may feel differently if I didn't know the purpose of what I am doing. Hence I ask Why? This question is open to interpretation but a good starting point would be "why is this question being asked? why does this requirement need to happen now? why are customers asking for this?" and so forth. Knowing the purpose, understanding how the idea was conceived from a business point of view is, in my opinion, very important. At this point, the data scientist is stepping into the business' shoes and seeing, perhaps for the first time, their perspective. Then we move on to the What? Remember earlier on I mentioned a section I used to read during my physics experiments - the Aim? Well, asking "what are we trying to understand? what result are we striving to achieve? what is the end goal that we are aiming towards?", gives us a broader perspective. The why showed us the starting point but the what should show us the finishing line for a scientific project.

However, there is another question I have the habit of asking because I am a scientist working with the data and, for the sake of the argument, not a businesswoman. I insist on asking How? I believe this question to be the cherry on the cake when painting the full picture, the final sketch before we begin the work:

How will the result be used?

The insight that this question will provide is incredible. Whatever the result should be, it has to conform and gain value with however it is going to be used. Whether it will drive an automated process or will help clients on the end of the pipeline to have better information at hand, it doesn't matter, as long as the data scientist understands how it will be used.

The process by which we begin to write the requirements should involve a lot of questions. The data scientist should pick the business' brain (in a manner of speaking), to fill in gaps that would not be obvious at this stage. It is noteworthy to highlight that the data scientist is asking questions to understand the business and not how to do the science. We are determining the Title of our experiment and defining our Aim. At this point, the data scientist and the business are discussing the potential hypotheses, or perhaps better referred to as gut feelings, that have been applied in practice by experienced business people but never proven with data.

A data scientist is expected to build a model that would deliver some result, and to do that, a scientist must know and understand the problem as if it was the scientist's question. The person juggling with data looking for answers must know what they are looking for and it must align with how it will be used. Having an in-depth discussion with businesses about the proposed project will save time on research, requirements and will build the foundations on which the entire science will be based.

Businesspersons are individuals who have a good understanding of their sector and have experience. Conducting research on the project by oneself will not be as fruitful as asking questions directly to the person putting the proposal forward. Doing the research after having asked questions, will lead to focused knowledge and relevant material. Bridging the gap between data science and business begins with one step, at the very beginning of a project. The definition of the project merits time and effort. Exchanging views, current beliefs but mostly experience, will give both parties a glimpse of what the other is looking for or what needs to be done.

There are occasions when the business doesn't know exactly what the target is and just wants insight. In that case, the end goal is simply new information and that usually cannot go wrong, unless it goes against their understanding. It has happened and will continue to happen for data stores secrets yet to be revealed. Other times, the business' goal is set and the data scientist should take advantage of this. The experience of the business is the most valuable, not because they can't be wrong data-wise, so to speak, but it provides the right direction for the research. Moreover, pushing for information will encourage the businessperson to clear out doubts or ambiguity that they may have and reach out to the data science team with additional information on the matter. Remember, the clearer the guidelines the better refined the research will be. Sometimes, it is also helpful for the data scientist to explain, very simply, some processes that could be used to tackle the project.

In the end, keep in mind that we are only humans. Despite the meetings, we still misunderstand and make mistakes. The question is how much room will be allowed for these to happen. As a scientist, I understand that one question can lead to different and distinct answers. Despite all these answers being correct, it does not mean that they are useful for what the business intends to do with it. Use the time to construct the hypothesis, for it will prove to provide strong communication, better collaboration across departments and progressive business decisions based on Data Science techniques.