Doing a survey or making a joke?

Dudi Dror, CEO of the Askria research company, reveals the gaps, biases and problems in the Ministry of Transportation's public transportation survey. The ministry: "The professional teams knew how to deal with deviations"

Dudi Dror, Askria

20/06/2022

Monday, 20 June 2022, 09:32 Updated: 09:53

• Share on Facebook

• Share on WhatsApp

• Share on Twitter

• Share on Email

• Share on general

• Comments

  Comments

Transportation is public, but of which public? (Photo: Reuven Castro)

In the last days of the Ministry of Transportation's large public transportation survey, religious sector WhatsApp groups circulated messages calling on the public to identify as secular and traditional, in a survey that examines, among other things, public attitudes toward public transportation on Saturday.

The goal: to create a presentation according to which many more secular and traditional people do not support public transportation on Shabbat at all.

If the Ministry of Transport wanted to conduct a survey that would truly reflect the position of the general public and without allowing certain population groups to create bias, it would have to choose another sampling method, which even if not completely prevents biases it at least significantly reduces them.

Opinion polls are the best source of information we have for evaluating public attitudes.

Their purpose is to gauge the attitudes of the general public, their views, beliefs, attitudes and expectations towards certain issues at a given point in time.

These surveys are conducted within the framework of clear methodological rules, in a small and representative part of the surveyed population, known as the 'sample'.

More on Walla!

Bad smells, overcrowding and noise: What makes the lives of passengers on public transport bitter?

To the full article

When these are the sectors proposed by the survey at the beginning of its activity, the direction is clear (Photo: screenshot, screenshot)

Properly conducted public opinion polls express public attitudes with good accuracy and a small sampling error (it is customary not to increase the error beyond 5%).

Whereas studies that do not adhere to a proper methodology may reflect various biases - including biases towards the opinion of researchers.

Amateur public opinion polls, also called 'straw polls', are polls distributed in a non-probability sample (ie one that does not allow equal probability for everyone to participate in the sample), have no scientific validity and often their results are not a reliable measure of public opinion. The general population.

In the case of the Ministry of Transportation survey, a probabilistic sample would not have prevented someone who would consciously choose to identify as secular while living a distinctly ultra-Orthodox lifestyle from doing so, but would at least prevent the possibility of a biased response by large interest groups.

Indeed, polling institutes are faced daily with the challenge of conducting a survey that accurately reflects the relevant positions and eliminating possible biases.

Such biases can stem, especially when it comes to influencing strategic issues such as elections and laws, from the interests of population groups seeking to influence as a group the survey results or for a variety of other reasons.

The research companies deal with biases in different ways depending on the type of bias, such as choosing sampling methods relevant to the question (telephone instead of internet), repeating sensitive questions in different ways throughout the survey and more.

The wording "weekends" is misleading, since Fridays and Saturday evenings are also included in the weekends, but religious ones use public transportation (Photo: Yotam Ronen)

Dudi Dror, CEO of Askria (Photo: Almog Gabay)

However, errors in the survey can also be in the wording of the questions themselves, which are not worded so as to accurately reflect the research question.

For example, in the question of the transportation survey "Will you use public transportation at the weekend", a secular would interpret a weekend today Saturday while a religious and ultra-Orthodox would understand that it is also a Friday and the answers - accordingly.

Also, the wording "weekend" is misleading, since for the religious traveler, the use of public transportation on Friday and Saturday nights is not a controversial issue, but these days are included in "weekend" as defined in the survey.

The wording of the same question (in the transport survey) "Will you use public transport at the weekend to consume essential services such as a hospital" could also be interpreted in a number of ways, as a hospital is an edge example of essential services and the inference from this example to essential services is limited.

It is also clear that in the event of an "essential need" there will also be no impediment for the religious public to use transport on Shabbat, and in any case, in a vital case, public transport will not be used at all in all sectors.

Religious people will also come to the hospital on Saturday for an "essential need," but the wording of the question is biased and misleading (Photo: Reuven Castro)

The survey as conducted, and although it has already been answered by over 100,000 respondents via the Internet, does not provide a credible and true picture regarding the positions of the general public.

Blocking the ability to fill out the survey from the same IP address is very easy to bypass.

Such biases could be eliminated using a smaller but probabilistic sample and one that does not allow anyone to respond except those selected for it.

Therefore, if the Minister of Transport wants to conduct a real attitude survey, which reflects the public attitudes as they are in practice and without biases to examine a sensitive strategic move like public transport on Saturday, you should plan the survey much smarter, with a real understanding of the target audience and especially without burying Your head in the sand and think that he is indeed representative.

Because right now as it is, any government decision that is based on it, will not really represent the general public.

More on Walla!

The data make it clear: Public transportation in Israel is heading for disaster.

And where is the Ministry of Transportation?

To the full article

Ministry of Transport response

We asked for the response of the Ministry of Transportation on the following issues: the number of participants in the survey and the distribution among the respondents frontally and on the survey site, for easily biasing possible answers and for misleading questions regarding public transportation on Saturday.

This is the firm's response that is fully presented:

the survey was launched three weeks ago, and ended this week with great success and above expectations, with about 150,000 respondents.

The survey is conducted, among other things, by dozens of surveyors in a wide variety of key centers throughout the country, with a wide spread in the various sectors in Israel, as well as online via the Internet.

The Ministry of Transportation has launched the large transportation survey, with the aim of inviting the public to assist in the development of transportation processes in Israel, while involving the public in decision-making processes.

The survey is intended to examine the public's attitudes regarding transportation in Israel, and to assist in determining policy and removing barriers, improving the transportation system and improving the ministry's activities.

Past experience in similar processes shows that the obligation to register for the survey significantly reduces the number of users and therefore was recommended by the professional bodies to conduct the survey anonymously, while limiting the possibility of responding once from each device.

Respondents can register at the end of the process to receive the survey findings after they have been analyzed.

It should be noted that even an identification system would not have prevented impersonation, and from the analysis of the information collected so far it appears that the vast majority of the respondents are not impersonating.

The findings of the survey will be analyzed by the professional teams in the Ministry of Transportation, who know how to deal with diversions here or there.

• vehicle

• Car News

Tags

• Public Transport

• Survey

• Department of Transportation