Comments
Follow Comments Sorted by time
cryoenthusiast
· 4 years ago
· FIRST
What do I have to do? A meta analysis of 30 different studies? I mean sure but that’s why we have researchers so they can do it for us.
guest_
· 4 years ago
Very true- but sadly things are seldom so simple. The first major problem I’ll mention is “peer review.” Often, lay persons and even scientists are likely to take “peer review” as some sort of Devine certification. The fact is however- there isn’t a singular operational definition of peer review. 50 scientists can look at a paper and how it was reviewed and come to different conclusions as to wether or not it was “peer reviewed.” How many peers do you need? Who are your peers? Other scientists, scientists in your field, scientists doing the same type of research and able to fully understand the subject with experience? Each one of these offers its own potential issues like biases or knowledge gaps depending on how your study reflects on their own work. And of peers.... in a technical sense and to be blunt- if you’re a poor example of a scientist, 1 or 60 other poor scientists validating a thing doesn’t really matter if they are all just... not very good or bright.
1
guest_
· 4 years ago
The next problem lies on studies. You can find article after article and discussion after discussion on the unreliability of studies- mostly do to flaws in the scientific process and the procedures and operations as well as practical realities. Studies are usually paid work for scientists. They are ways to get grant money or salary, and publishing them helps build a name and recognition. Peer reviewers usually aren’t paid. Their “compensation” comes in the form of Recieving recognition for their review on a published study. The systems of tenure and funding and their own careers and earnings for scientists are tied together in a flawed system that encourages the “ambitious” scientist to get out and get their name published.
▼
Show All
guest_
· 4 years ago
It goes deeper- I won’t go too deep because there are TONS of papers and sources for more information via search. But the next problems concern bias. Studies are most often funded- often by private entities. While it’s uncommon to just “buy” a study in the terms that you set out explicitly the results you want and how much that will cost- there are powerful motivations and often powerful subtle and less than subtle pressures on a scientist working for an external and biased patron, to not necessarily “fake” a study- but to perhaps “massage” parameters or data or conclusions to be suit the purpose of the commission of the study. If a patron doesn’t like what a study says- they often can suppress it or use other means to effectively get the result they want even if the scientists involved don’t play to their game.
guest_
· 4 years ago
One such example is multiple studies. We can re do a study, such as a study on random samples, multiple times and treat each data subset as its own study, and do this until we get a group with favorable data to what we want to see. Likewise- if no single study gives favorable data, we can combine the data of the studies into one large group and then, after adjustments for variance etc- can produce favorable data.
guest_
· 4 years ago
The flip side to that- is that often studies aren’t repeated. Not by the group doing them or by other groups. Sometimes you won’t even find anyone else who has done the same study or a study on the same subject. Having 15 studies by different teams with different sources of funding that aren’t related but happen to be on the same subject- can lend credibility of all or most of these studies come to similar results.
cryoenthusiast
· 4 years ago
isnt that a fine line though? at what point is does statistical analysis become statistical deceit?
1
guest_
· 4 years ago
A good way to put that might be an Amazon or Yelp review- looking at reviews is a sort of “study” most people can relate to. One person will simply look at 5 similar products and whichever has the most stars they will conclude is best. Another person will read the reviews and see what people said- and factor in why they gave it X stars- and another will do both those things before making their conclusion- but also look at how many reviews there are- four 5 star reviews or 200 reviews that average 3 stars- which is better?
1
guest_
· 4 years ago
And this illustrates subjectivity in a way most people can relate to in data. When you’re reading reviews for a product or place to eat- if you see a review that’s stars and no text- do you ignore that review or include it to your decision? If you have a 1 Star review that just says “sucks” do you include that? If you see a bunch of 1 Star reviews on a product and they say “it took 4 weeks to get and Fed Ex lost it for awhile ...” do you include those- even thought shipping time has nothing to do with the products quality, and the seller doesn’t have control of fed ex? Things like that can lead 2 people to look at the same data and draw different conclusions.
1
guest_
· 4 years ago
Business and stock markets are a perfect example that are harder for many lay persons to relate to directly- but experts of their field with proven track records in earnings success can look at data on these things and come to very different conclusions. And that too is a part of studies. Not just that scientists can view the results and come to different conclusions, or that lay persons can interpret those conclusions to mean different things- but that many people only read the conclusion of a study- the opinion of the data- without validating that the data presented would match the conclusion. A non expert may not have the ability to make that call- if they did we wouldn’t need conclusions at all. But- without standardized practices in peer review, without being able to compare to a body of similar studies by other researchers- we are left with ambiguity in deciding to trust a statistically unreliable medium.
guest_
· 4 years ago
Conclusion and Tl:dr- there are lots of issues with science and studies and their integrity. There aren’t really universal standards of burden of proof or interpreting data that we can say are followed and are agreed upon. There isn’t a singular logic and a process of documentation and certification to show these burdens were met. Like many things- it’s far from perfect but it is the best we have, so I’m not saying science can’t be trusted or experts can’t be. The problem is knowing when to trust them or when not to- and that becomes its own subjective can of worms.
guest_
· 4 years ago
When it comes to citing studies- it’s probably best to do ones due diligence or at least be open to the possibility that such studies may be contradicted by evidence and not correlate a study with concrete fact. Studies are an indication of what could be true- supporting evidence to a position by by themselves they are not adequate support and need to be braved by additional studies of the type and other forms of evidence. But it is best to stick to sources for studies which are as reputable and of known and trusted sources; have the least evidence of bias or possible bias if one wants a study to be taken seriously as evidence.
guest_
· 4 years ago
@cryoenthusiast- that’s a good question. I don’t really have an answer. Like so much in data- it’s subjective and not universally defined. I’d think that the line itself falls in a place dependent upon the specifics of each circumstance. Factors like criticality, scope, risk, impact, etc. of the actions we are relying on the data to inform; as well as the urgency of the thing. Questions like “what was where the universe is before the universe existed” don’t seem to imperative to answer quickly- so we can mull that one over for a few millennia at least before needing to call a vote. That which is more pressing to form an opinion- we are compelled to be less thorough.
lucky11
· 4 years ago
I know you kinda touched on it but an important aspect for us non-scientists is the initial premise/question. Is it a study to determine what happens when x meets x or if x happens? Or, is it a study that goes x is happening, this is a verifiable fact, now why on earth is it happening? Depending on the type it does make it easier to decide whether to believe the results depending on things like sample size and what not.
1
cryoenthusiast
· 4 years ago
All of this is so exhausting. Having to constantly make decisions that require serious knowledge while at the same time having to counteract the effects of those who want to take advantage of decision fatigue and trust. Can’t have shit in in this Information Age.
1
guest_
· 4 years ago
@lucky11- you bring up good points. Its not just about who the study is by, it’s about how the study goes about its goals and is stated as well. A nuance that is often lost on people- things like statistical correlation get conflated to causation. Things that indicate a higher instance of X among Y get read to mean that Y represents X- and of course- a study that sets out to “prove” a hypothesis with data vs a study that intends to view data to see what the data indicates... the former is biased from the start as the goal isn’t to gather information to see what it indicates- but to gather information which proves what one already thinks.
1
guest_
· 4 years ago
@cryoenthusiast-damned if you do and damned if you don’t. The problem with information- let me be clear when I say enough information exists to not only track every action of a good deal humans on earth- but even to predict future actions. There’s just so much data that (as far as we know) no one has the tools and bandwidth to gather and make use of it. If you think of looking up questions or tutorials online- you might search hours for days and find nothing- then randomly come to or be sent a link with exactly what you wanted but couldn’t find. Often times the answer to questions is out there and we just don’t know where to look! Much like some piece of history found in a thrift shop that historians have sought for ages- it was right in one place but no one knew where to look.
1
guest_
· 4 years ago
With so many people using information and disinformation as tools in their agendas- or pushing an agenda using a mask of science to hide it- it’s tough. But before all this info- what’s the difference? Media and government and such still lied and used control of information- before people didn’t have the information to question BS as BS and mostly took the BS as fact. Now people just have a choice of what BS to believe or confusion at being unable to pick a flavor of BS. The end result is still BS.
1
guest_
· 4 years ago
I mean- to be honest, I often think one solution (along with reforms and demanding higher standards from media etc) is simply... for people to stop having so many opinions. In 1902 most people didn’t know or care what was happening thousands of miles away- and largely if they were effected by it at all- knowing wouldn’t change a thing. If a person chooses to ignore the news and all that- how does their life change?
1
guest_
· 4 years ago
Donald Trump makes a lot of people feel one way or another- but many people- like myself- can say that in general- if it wasn’t for news and talks of the man- my life would be largely the same as it was under Bush, Clinton, Obama. What’s the guy really done that isn’t subjective? Some people have directly benefited and others have directly suffered.
1
guest_
· 4 years ago
But if you couldn’t tell the difference without knowing... does it matter? Now I believe people should try and stay informed and engaged- but.... do people really need opinions on everything? Is it so shameful for an individual to say: “I don’t know enough about the issue or the subject to give and informed view...” I think honestly- that’s the part of the problem that impacts most of us- when people who have no qualifications to be involved jump in.
2
guest_
· 4 years ago
Opinions- “I think equality is important but I also think most police are good and do a good job and need support...” there we go. If one doesn’t have an encyclopedic knowledge of sociology and US history and law and blah blah- to be qualified to make a statement... they can express an opinion without getting into details they may know little or nothing about.
guest_
· 4 years ago
And if we each focused on improving things we see around us- we could really make a difference and not need to be in everyone’s business unless they were abusing human rights or constitutional freedoms more or less. We could give our opinions on what we value and let the representatives we elect represent those for us. And when they don’t... we could fire them and elect someone who will instead of propping them up because we are ideologically aligned even if they don’t actually deliver what is promised or required of their office.
1