A quick guide to information evaluation

[Jenett]

(I blame Darkhawk for nudging me to do this in the best way possible. It might have gotten a little away from me. A version of this first post will be up on my Seek Knowledge, Find Wisdom blog on Tuesday, possibly after a bit more editing.)

This is by no means complete, but should be a starting place, and there are some handy links at the end. Please feel free to discuss other aspects of information evaluation, give other examples, ask questions, and whatever else seems relevant to the topic!

Basic principles:

• We all have biases and things we know more about than others.
• Some people are more up front about this than other people.
• Ditto goals. We all have them, some people are more up front about them.
• Be really suspicious of the people who claim they have the absolute truth and are telling you for your own good.

(They probably don’t and they probably aren’t. Especially if you don’t have a preexisting trusting relationship. Real world stuff has fewer absolutes, for one thing.)

Who is this person (or What is this source?)
Start with the basics. Who’s telling you this thing? What’s their background? If it’s a website without an individual author, what do you know about the site?

You may need to file this in “Need to do some more research” but knowing you need to do that is a great first step. First thing: check out the ‘about’ page, or a bio. Usually this will give you some hints on what they’re about and what they care about most.

If you’re not sure where to start with that, try searching the person’s name (plus maybe a term from the topics they write about, if you need to narrow it down) or search on the name of the site. Sometimes adding in words like ‘review’ or ‘about’ will help.

Even just knowing what kind of source this is can help. Personal website? Newspaper that’s actually well-known and reasonably respected (even if you don’t agree with them)? Pocket of internet culture you weren’t previously aware of? Political group hidden behind astroturfing techniques?

I sort things into “Probably reasonably competent”, “Dubious” and “Need more information”, personally.

Probably reasonably competent sources are those I’ve checked out before, and came up reasonably well sourced. I still need to check the specifics here, but they get some starting benefit of the doubt. Dubious sources are those that have come up short before. Everything else gets filed in ‘need more info’.

What are their goals?
Education? Information? Sell something? Share something gorgeous or fun or amusing? Are they trying to persuade you of something?

What do they get out of you believing them and taking them (or their information) seriously? Are they being up front and honest about that?

Here’s an example: sales sites are not the most fun thing ever, but there is something refreshingly honest about “Buy this thing from me and here’s why.” It’s clear what the people want, and usually pretty clear what’s involved in getting it.

On the other hand, a lot of sources in the political realm are trying to persuade you of things, but it’s not always clear what they’re trying to persuade you of. (Or whether they’re not trying to persuade you at all, but are instead signalling to their core base what they care about.)

This is often where you see a lot of vagaries and unsourced information that plays on emotions rather than treating you like the intelligent, thoughtful, considerate person I want to think you are.

Where did they get their information?
This is where we get to the meat of things. People who are saying trustworthy things should give you a way to check, or more information about how they know that.

When we’re talking to a friend, we put what they tell us in the context of all the other things we know about them. They’re reliable as anything with a ride when it’s important, lousy at getting stuff to the post office.

They have a lot of specific experience in dealing with Mercutian rabbits, and the last fifty things they told you about those rabbits turned out to be right, but they’re not nearly so reliable about Venusian wombats. And they’re normally great about Saturnian leopards, but there’s this one weird quirk, don’t trust their grooming recommendations.

When we’re reading a random website, we don’t have that. We can’t put some of what they’re saying in context without more information.

That’s why their sources matter. Do they tell us where they’re getting their info? If it’s unnamed experts and sources, be dubious. (Though there’s a link below with some more about how to evaluate this with more nuance.)

If they claim specific expertise, can you verify that or does it seem in line with what someone with that expertise would say? (If someone claims to be a lawyer or doctor or librarian and says stuff that is way outside what you’d expect, be dubious without more specifics. Maybe a lot more.)

When is this information from? Is this a topic where currency matters a lot? Some topics change fast, some don’t. Sometimes the info that debunks a current thing has been around for a while (so older info may still be helpful in sorting this out.)

What kind of source is this, and is the information presented in a way consistent with quality information in that kind of source? Reputable newspapers don’t generally go in for explicit personal insults or completely unverified sources. (Unless they’re quoting someone who used one.) Less reputable current events sources might.

Expect better of where you go to learn things. If they’re not giving you meaningful information, go to sources that that will. You can do better than speculation and gossip.

Other key tips

Beware of absolutes, especially in complex situations.
There just aren’t that many absolutes in the world. This is especially true when looking at expert statements: few experts will give 100% certainty. If they do, they will likely also be explaining why. Look for that explanation.

If a media source says something absolute, check into what the experts actually said, and what information they looked at to get there. Chances are pretty good the expert was not nearly so absolute about things.

Be dubious of things that are too good to be true, too weird, or too perfect.
Again, the world just isn’t like that very often. The more we realise that we live in a world that has a lot of shades of colour and nuance and different experiences in it, the sooner we’re going to get better at evaluating information effectively and using it well.

Is this a situation where there are strong emotions?
Sourcing is often not the top priority in these cases. Which is understandable, but just because someone’s having emotions all over the place doesn’t mean you have to use everything they tell you as the basis of your decisions.

Emotions don’t mean someone’s wrong, mind you.

It is, for example, pretty reasonable for someone to be emotional about a topic that has a major impact on their daily life, health, safety, family, or religion, if other people are treating it as a purely intellectual discussion. But a story that’s playing on your emotions to make you feel upset or riled up or righteously victorious, you should be suspicious of that.

If emotions are in play, and you’re not in the middle of the discussion, it’s usually better to pause and take a moment to look at what’s being said.

Who has real experience with this thing? Who doesn’t? How does what people are saying match up with other kinds of information you can find or your experience of people or situations? Who has what at stake? Is this a real person who has specific experiences, or is it a made up storm of emotion that’s trying to get you to react a certain way?

Some additional resources:

Here are a few additional links worth reading

• A guide to evaluating fake news from NPR’s All Tech Considered
• Library guide to evaluating fake news from Indiana University East (Has some great specific examples and walkthroughs)
• FiveThirtyEight’s guide to anonymous sources and what they mean (I found this very useful in figuring out how to weight some things I’ve read)
• LifeHacker talks to two experts about evaluating information. One of them is Phil Plait, who’s great at exposing bad science.

This is only a beginning - there are lots of nuanced issues involved in how we find and evaluate information I haven’t even touched on here (like who decides what gets researched that you can refer to later.)

[Darkhawk]

(I blame Darkhawk for nudging me to do this in the best way possible.

It is very kind of you to provide me with a hook point for something I want to immediately hijack to talking about a slightly different thing!  Specifically, evaluating various (especially pagan) religious groups.

Which is tricky because the hard factual information is a bit more fiddly in matters spiritual.

So, religious groups.  As was discussed in a thread recently (about "drawbacks to your religious experience"), a lot of people wish they had them, and there are problems with finding them, but there are also drawbacks to having them.

So a lot of these discernment and evaluation things are handy to apply to groups.

Who is this person (or What is this source?)
Start with the basics. Who’s telling you this thing? What’s their background? If it’s a website without an individual author, what do you know about the site?

Who is running the show?  What credentials do they have?  What status do they claim?  What backs up that status?

When was the group established?  How stable is its membership?  How stable is its scheduling?  What resources does it expect members to dedicate to its upkeep?

In the case of a witchcraft tradition, does the group leadership have a lineage and a willingness to offer vouch?  If that lineage is in some way questionable or out of the ordinary, are they up front about that or is that something that comes out while digging?  If this group is not lineaged, where do they get their stuff from?  What is the quality of their source materials?  (A bootstrap eclectic working from DJ Conway and Silver Ravenwolf is going to produce... different content than a bootstrap eclectic working from Buckland and Starhawk, is going to produce different content than someone working from public works of Gardner and Crowley.)

In the case of a reconstructionist group, what are the sources being used?  Are the origins of rituals and liturgies and theological concepts things that people are encouraged to look up and see for themselves?  Alternately, is everyone expected to read the same books and come to the same conclusions because I researched it uphill both ways in the snow you whippersnapper, you can do the same?  Does anyone have a formal educational background in something relevant to the topic?  What sort of deference is that formal educational background expected to get?  Is it required to have that formal educational background to become a person of importance to the group, or are there other ways to contribute?

In an eclectic or multi-tradition group, how much set liturgy and practice is there?  Are newcomers expected to assent to the established protocols, or do they have influence on how rituals or events are conducted?  Is that group aware of the variety present in the pagan movement, or are they the sort of people who are going to throw a shitfit if you happen to point out that Samhain is not 'the pagan new year'?

Does anyone in the group claim special status, powers, or background?  (I mean, not everyone is as ridiculous as the person who claimed to be the half-vampire personally responsible for training Anubis up into combat trim now that the gods have returned for the War on the Astral.)  Do they present this as a quirk or do they expect to be given extra respect and standing because of it?  Divine rulers, fairy royalty, hereditary high priestesses, and so on really are a dime a dozen; is this one in particular credible?  Is believing in their status critical to getting along here?  Are they going to pull a power grab like the person who tried to forbid a group member from transitioning because in their last incarnation on the homeworld they were their assigned sex and how dare they mess with that?

If there is a homeworld (or whatever other background information) presumed for members of the group, are those members able to find out their place there themselves or is it assigned to them by leadership?  If assigned by leadership, is there a standard protocol for doing so?  Where did that protocol come from?  Is it conducted by something like divination, by trance induction, are there verifications and controls?  If dealing with, say, 'which of a company of gods, spirits, and other entities are closest to you', what are the protocols for dealing with other beings, either in the system of the group or outside it?

What commitments does the group expect a person to make?  Do they get more onerous over time?  Is exclusivity one of them?  Are those commitments laid out in advance, or is it more like "we expect you to adhere to a conservative moral code" without anywhere publicly stating what that means?

What sort of control over members does group leadership exert?  Does anyone ever say "you are obligated to not associate with that person, read that forum, go to that club, if you are a member here"?  Are there grudge matches ongoing with individuals or other groups that you will be expected to align with?  Do the stories that group members tell about non-members match up with information more directly from the "other side"?  How invested is the group about those other groups?  (Does the training regimen include, "Define a cult, and explain how other-group is one and we aren't?"  For a particularly gloriously blatant example.)

What is the history the group and its leadership has with disprivileged others?  (Paganism as a whole has a substantial race problem, and that's worse in some places than others.)  Is any of the group's material appropriative of minority cultures?  Do they make excessive claims about indigenous status or apply creepy emphasis on the importance of ancestry?  What are its gender-related politics?  Does anyone assert that men can't hold leadership roles, or that women can't?  Is gender essentialism critical to the group's theological structure?  Does that essentialism carry over into non-ritual behavior?  Do people freely spout nonsense about chronic illness or disability, or deprecate the use of accessibility aids?  Are meetings held in an accessible location?  Is there a presumption of a certain level of wealth required to participate?  (Those crystals add up.)  Are there queer, agender, androgynous, multi-gendered spiritual figures in the religion as practiced?  Is there space within the official structure for people who do not cleanly fit categories?  Etc.

Um.  I wandered from individuals to groups but I think that makes sense as a progression.

What are their goals?
Education? Information? Sell something? Share something gorgeous or fun or amusing? Are they trying to persuade you of something?

Religious groups may be founded for a variety of reasons, which include (and are not limited to because it is late, I am drunk, and I am not omniscient even when sober):

* fellowship (social gathering)
* shared veneration of the same entity group
* community gathering and community ritual
* establishment and working within a familial setting
* shared ritual in a particular tradition
* cultivating, preserving, and living within specific religious mysteries
* recuperation from the strain of the mundane world
* mysticism, trance, and ecstasy
* instruction in a particular tradition/religion/path/thing
* ritual or magical work towards a specific set of goals

When looking for a group, it is worth evaluating what of those goals one has, and what of those goals the group is interested in.  A good, solid, respectable group that's basically a reconstructionist book club is not going to be a good match to somene looking for an ecstatic drum circle.  Best to find a good, solid, respectable drum circle for that.

What do they get out of you believing them and taking them (or their information) seriously? Are they being up front and honest about that?

So a shout-out to the Bonewits Cult Evaluation grid here.  Some of which I touched on above, really.

Obviously, a lot of people will leap to financial stuff here - pay for training is a big one about which many debates are had.  But there are other things.  Some groups will engage in what's called "love bombing" behaviour, treating potential new recruits as super-special amazing people who are privileged to join the secret club, basically.  It can be tempting and seductive, and often comes with a seamy underbelly.  Exploitation is not merely financial; being the one who's always bringing the big dish to the potluck, or always has to do a little extra driving for the carpool, or any of a number of other things, can be a problem people get stuck in.  A lot of groups fail because someone got burned out on doing the administrative work and nobody else pitched in - they were happy to have a group, but not enough to throw in some effort to make the group work.

Does the group get some sort of status from numbers?  Are they engaged in an ideological turf war in which more people on "their side" count as victory points?

And back to theology and where people get their ideas from: are those ideas sourced, and from where?  How much scope for disagreement on interpretation is there?  (Keep in mind that it is common in many traditional religious witchcraft groups to have a dozen different interpretations of the nature and significance of deity among the six to eight people present, none of which are relevant to gathering to do religious stuff.)  Is the group fixated on a theological concept that basically someone made up on a boring Tuesday afternoon and which is now being put forward as Profound Actual Spiritual Fact-truth?  Does the adherence to that concept jive with the group's stated position on UPG vs. historical resources vs. living tradition?

What sort of transformation is expected of group members?  Obviously, in an initiatory tradition there is preparing for initiation and the changes that come of that.  However, actual adherence to a spiritual discipline of any sort will have ripple effects.  What do those ripple effects look like?  Do they create a world more worth living in, or do they seem to do harm?  Do they encourage group cohesion at the price of generating hostility to outsiders?  Or, alternately, are the members of the group mostly looking to preserve their peace of mind (like the whiny white Buddhist who was handwringing about how all these minorities speaking up in UU churches were harshing his mellow, he didn't come to religion to have to confront injustice, he came for sweet, sweet comfort).

Does the group claim knowledge of an absolute spiritual truth that ought to be believed by every person (every pagan, every adjective subgroup, etc.)?  How do they deal with those who do not share their belief, or have opposing beliefs?  Is the group dependent on the demonisation of others in some fashion?  (Another denomination of the same religion, people who don't read the particular Big Name Pagan's books, Satanists, followers of the wrong entities in their cosmological structure, etc.)  Alternately, does the group tend to dismiss differences with "all roads up the same mountain" or similar philosophies?  Are members of the group in need of external validation and affirmations from newcomers that they're doing the right thing?  Do they feel the need to convince others to see it their way?

On the other hand, a lot of sources in the political realm are trying to persuade you of things, but it’s not always clear what they’re trying to persuade you of. (Or whether they’re not trying to persuade you at all, but are instead signalling to their core base what they care about.)

Oh so double true in the religious realm.

Where did they get their information?
This is where we get to the meat of things. People who are saying trustworthy things should give you a way to check, or more information about how they know that.

I am a huge fan of footnotes.  When I have my recon hat on I want all the footnotes.  A recon group that can't lay hand on which scholar or interpretation they're deriving something from or admit that that's something someone made up along the way or note that it's a blend-in from somewhere else is a recon group with problems.  It is not at all uncommon for recon groups to develop some sort of in-group jargon or weird interpretations of academic material and then take that stuff to scholars - who wind up staring at them blankly because that is stuff someone made up, usually based on a weird read of something or a wild extrapolation, and its connection to the academic understanding of the topic is basically nonexistent.

Similarly, I've seen a number of  groups work with, for example, African Diaspora societies to obtain skills and understandings involving stuff like trance possession - and some of them will talk about where they got their stuff and how much of it is basically straight up tech translation, while others won't.

Back in the 70s and time on either side of it it was super-common for people to cross-train and produce synthesis traditions when they'd done so.  It's one of the reason the family history of religious witchcraft is such a glorious mess - it's basically a giant heap of religious fornication.  I think these days that sort of cross stuff is not really well documented, not in the sense of someone being able to say 'I trained in X, Y, Z, and what I do is mostly Y, with my own twist on it, and influenced by A, B, C'.  Elfwreck, who used to post here, commented several times that the religious witchcraft scene in the US was probably a royal mess because people thought Buckland, Cunningham, and Starhawk were all talking about the same religion.  And the Frosts.  Laurie Cabot.

(http://sunsinherbranches.net/wp-content/uploads/2016/10/RW-Soupmap-v.2.1.png - Buckland, upper left, three lines of descent: Seax-Wica, his books (to the extent that's not Seax-Wica), and his initiates; Cunningham, lower left, influenced by him (I *think* I filtered all the initiates through the initiate bubble), but also the Majestic line of CVW and Grimassi; Starhawk, lower right, in the blue bubble zone of Feri derivatives rather than the green bubble zone of Wicca ones.  Frosts, center left, mysterious initiation story, druidry, and ceremonial magic.  Laurie Cabot, sort of on the right side of the middle, past life story, Celtic revival.  Et cetera, et cetera, and so forth.  All of this got compressed into a vague blodge popularly called 'Wicca' even though that makes no fucking sense.)

So, you know, is someone selling you 'Wicca'?  Do they know what that is?  What sort?  (See also Jenett's people mean a lot of different things when they say Wicca page - which one are they using?  Is it the one you want?

If someone's selling something else, likewise, check the thing.  Maybe you can't get a family tree of what all fornicated to make the thing happen but that's good to know.

When is this information from? Is this a topic where currency matters a lot? Some topics change fast, some don’t. Sometimes the info that debunks a current thing has been around for a while (so older info may still be helpful in sorting this out.)

Historically-interested religious groups in particular have a fascinating habit of latching on to the current state of the understanding of something - history or biology or some other thing - and locking it in stone.

And there are good ways and bad ways of handling it.  We used to have a poster here (her memory a blessing) who was a recon in the mode of a scholar who was long debunked, but was of the opinion that the rituals worked, consistence was important, the people were happy, the gods were happy, and that was much more important than chasing the state of the art on historical accuracy.

(If the state of the art on historical accuracy matters to you, reconstructionist religion is a terrible idea.  The parts that are knowable are infinitely interpretable, and you will often have to pick one rather than live comfortably with the argument.  Academia is pretty much about the argument, not the picking one, and will not save you.  Nor will it stop reality from someday throwing in some evidence you weren't looking for that wrecks everything, forcing you to choose between the state of the art and the functioning sytem yet again.)

What kind of source is this, and is the information presented in a way consistent with quality information in that kind of source? Reputable newspapers don’t generally go in for explicit personal insults or completely unverified sources. (Unless they’re quoting someone who used one.) Less reputable current events sources might.

Beware of absolutes, especially in complex situations.
There just aren’t that many absolutes in the world. This is especially true when looking at expert statements: few experts will give 100% certainty. If they do, they will likely also be explaining why. Look for that explanation.

How firmly and absolutely are things presented?  How flexible is the tradition?  Does it take into account new developments in unrelated fields which nonetheless have repercussions?  When there are firmly held positions - often on things like purity, appropriate or inappropriate offerings, and so on - how does the group deal with people who violate them?

Does the group fall into the modern error of mythological literalism?  How does it interpret sacred stories?  How are those stories told, ritualised, or otherwise referenced?

Be dubious of things that are too good to be true, too weird, or too perfect.
Again, the world just isn’t like that very often. The more we realise that we live in a world that has a lot of shades of colour and nuance and different experiences in it, the sooner we’re going to get better at evaluating information effectively and using it well.

Back to the person who just wanted church to be about thinking he was a good person.  If it doesn't require doing anything, being anything, changing anything, it is probably too good to be true.  Religions as a rule do not provide an excuse to sit on one's ass all the time and watch the world burn.

Also lovebombing.  Also secret special traits.  Also only the bestest specialest people join here, and you are one of them!

Like I said in my evaluating your UPG thread a while back, there's a difference between wish fulfilment and "... well fuck that's going to be a lot of work."

OKAY I have made MANY WORDS at Jenett who starts with many words.  I am shut up now and to to bed like someone who is sober and reasonable and has to be up in the AM.

[Faemon]

Where did they get their information?
This is where we get to the meat of things. People who are saying trustworthy things should give you a way to check, or more information about how they know that.

(...)

Expect better of where you go to learn things. If they’re not giving you meaningful information, go to sources that that will. You can do better than speculation and gossip.

Well said!

I personally make a maybe persnickety distinction between "evidence" and "proof".

Evidence to me being a physical material for example, an iron ball dropped into a vat of clay from a specific height...Proof, being a series of statements that attribute meaning to the material. Or, a series of statements used to convince another logistician that the concluding statement is true, especially relating to the evidence (I should hope.)

The conclusive statement that the velocity of a moving object involves an exponential calculation where previously there were a multiple, continuing the example, would have from examining the material evidence.

But I wouldn't take it as accurate to point to a ball embedded in clay and simply say, "That proves it." That evidences the concluding statement, not proves the concluding statement. Proof would require a series of statements accurately reflecting on the consistency of the material, the conduct of the experiment, the control of the variables, and describing how the material evidence confers the conclusive meaning.

Proof might even turn out to be inconclusive, if another logistician isn't persuaded and argues so within the confines of logical guidelines, that the existing "proof" is too specious to prove anything, or the premise is faulty.

In the example I gave, it wasn't a specious faulty proof, except that I mentioned the experiment was conducted with iron balls. My source said they were lead balls. (Einstein's Big Idea, 2005 Nova documentary... I made a drinking game over the slow-motion strutting of the actors in the dramatic reenactments.)


I want to say that there must be a similar sort of experience-interpretation-conjecture-evaluation-peerreview process with regards to belief systems, regarding a body of info or piece of info as an evident thing, and checking one's own approach to it (and thought process, or meaning attribution process) as another.

[Yei]

Basic principles:

• We all have biases and things we know more about than others.
• Some people are more up front about this than other people.
• Ditto goals. We all have them, some people are more up front about them.
• Be really suspicious of the people who claim they have the absolute truth and are telling you for your own good.

(They probably don’t and they probably aren’t. Especially if you don’t have a preexisting trusting relationship. Real world stuff has fewer absolutes, for one thing.)

Who is this person (or What is this source?)
Start with the basics. Who’s telling you this thing? What’s their background? If it’s a website without an individual author, what do you know about the site?

You may need to file this in “Need to do some more research” but knowing you need to do that is a great first step. First thing: check out the ‘about’ page, or a bio. Usually this will give you some hints on what they’re about and what they care about most.

If you’re not sure where to start with that, try searching the person’s name (plus maybe a term from the topics they write about, if you need to narrow it down) or search on the name of the site. Sometimes adding in words like ‘review’ or ‘about’ will help.

Even just knowing what kind of source this is can help. Personal website? Newspaper that’s actually well-known and reasonably respected (even if you don’t agree with them)? Pocket of internet culture you weren’t previously aware of? Political group hidden behind astroturfing techniques?

I sort things into “Probably reasonably competent”, “Dubious” and “Need more information”, personally.

Probably reasonably competent sources are those I’ve checked out before, and came up reasonably well sourced. I still need to check the specifics here, but they get some starting benefit of the doubt. Dubious sources are those that have come up short before. Everything else gets filed in ‘need more info’.

What are their goals?
Education? Information? Sell something? Share something gorgeous or fun or amusing? Are they trying to persuade you of something?

What do they get out of you believing them and taking them (or their information) seriously? Are they being up front and honest about that?

Here’s an example: sales sites are not the most fun thing ever, but there is something refreshingly honest about “Buy this thing from me and here’s why.” It’s clear what the people want, and usually pretty clear what’s involved in getting it.

On the other hand, a lot of sources in the political realm are trying to persuade you of things, but it’s not always clear what they’re trying to persuade you of. (Or whether they’re not trying to persuade you at all, but are instead signalling to their core base what they care about.)

This is often where you see a lot of vagaries and unsourced information that plays on emotions rather than treating you like the intelligent, thoughtful, considerate person I want to think you are.

Where did they get their information?
This is where we get to the meat of things. People who are saying trustworthy things should give you a way to check, or more information about how they know that.

When we’re talking to a friend, we put what they tell us in the context of all the other things we know about them. They’re reliable as anything with a ride when it’s important, lousy at getting stuff to the post office.

They have a lot of specific experience in dealing with Mercutian rabbits, and the last fifty things they told you about those rabbits turned out to be right, but they’re not nearly so reliable about Venusian wombats. And they’re normally great about Saturnian leopards, but there’s this one weird quirk, don’t trust their grooming recommendations.

When we’re reading a random website, we don’t have that. We can’t put some of what they’re saying in context without more information.

That’s why their sources matter. Do they tell us where they’re getting their info? If it’s unnamed experts and sources, be dubious. (Though there’s a link below with some more about how to evaluate this with more nuance.)

If they claim specific expertise, can you verify that or does it seem in line with what someone with that expertise would say? (If someone claims to be a lawyer or doctor or librarian and says stuff that is way outside what you’d expect, be dubious without more specifics. Maybe a lot more.)

When is this information from? Is this a topic where currency matters a lot? Some topics change fast, some don’t. Sometimes the info that debunks a current thing has been around for a while (so older info may still be helpful in sorting this out.)

What kind of source is this, and is the information presented in a way consistent with quality information in that kind of source? Reputable newspapers don’t generally go in for explicit personal insults or completely unverified sources. (Unless they’re quoting someone who used one.) Less reputable current events sources might.

Expect better of where you go to learn things. If they’re not giving you meaningful information, go to sources that that will. You can do better than speculation and gossip.

On this, as someone who studies history, often with biases sources, I find myself thinking of ad hominem. I know that ad hominem is generally regarded as a fallacy, attacking the source of an argument, not the argument. But as a historian, knowing who makes a claim, and what their motives are, is an entirely logical thing to do. Its just obvious. People have agendas, biases, and are sometimes ignorant. And I'm not the only person to have noticed this; I think a few philosophers have also pointed this out.

My best guess is that the ad hominem prohibition developed from the context of a 'formal' debate (whatever that may mean in practise), where both parties are assumed to be neutral presenters of an objective argument. However, this does not apply in most circumstances, where a person's motivation, interests, and objectives, play a huge role in the way they present information.

BUT ...

I think we've all seen people's arguments dismissed because of irrelevant personal attacks (quite consistently in recent months). It makes absolute sense to prevent such attacks from being a part of debate and argument analysis. The hard part is knowing where the line is.

[Vixen]

You are brilliant, as always, Jenett! Thank you so very much.
While taking my own slow steps into American life with its wasp nests of political convictions, varying views on religion and wildly contradicting news reports, I could use a little bit of guidance.

What would you say are the best independent sources to keep up to date on latest American news?

[Jenett]

(Darkhawk, I would like to point out that you produced nearly twice as many words as I started with, which I find hilarious, given our usual history of me being wordy and you being pithy. But they are excellent and useful words! Keeping my comments to specific points, because otherwise this is an expanding lake of words that will fill up the universe, clearly.)

In the case of a witchcraft tradition, does the group leadership have a lineage and a willingness to offer vouch?  If that lineage is in some way questionable or out of the ordinary, are they up front about that or is that something that comes out while digging?

This is also one of those useful to mention cases of 'not all information is necessarily available to the randomly curious investigator' which is a complicating factor in research.

(A lot of trads won't give specific lineage info unless and until you're seriously being considered as a student or prospective member. For one thing, it involves some amount of outing other people and providing private contact info in many cases, and in others, the info is not necessarily terribly helpful to someone new to the community.)

But the point that the legit folks will be up front about how stuff works for them, and what that means (and usually give you some methods of figuring out if you want to keep talking to them until you get to that point.)

Does anyone have a formal educational background in something relevant to the topic?

Worth noting explicitly that academia expects one to specialise pretty heavily at the higher levels: a degree in the general field doesn't mean that someone knows details of even something that seems pretty closely related. Also, was that a degree they got 30 years ago and they've been working in an entirely different field, or is it something they've been actively keeping up with?

What is the history the group and its leadership has with disprivileged others?

To this list, I'd also add "Do the group's practices have reasonable accommodation for people who have a range of different needs and are at different places in life?"

It's pretty reasonable for a group to say "The stuff we do requires both skills and trust in each other: that's hard to do if people aren't here regularly." and lay out a schedule, and explain what stuff is most important, and what stuff is less important (though there might be consequences if you decide to go the movies that day instead of the group stuff.)

But a group that requires a dozen hours a week of face to face time is going to be harder for a lot of people with full-time jobs (especially not 9-5ish schedules) or children or who are working on additional education. And a group that yanks the schedule around without warning is going to be a lot harder for people whose lives are already a complex balancing act (or who are juggling health issues or childcare scheduling, or a job with different shifts)

What sort of transformation is expected of group members?  Obviously, in an initiatory tradition there is preparing for initiation and the changes that come of that.  However, actual adherence to a spiritual discipline of any sort will have ripple effects.

One thing I talk about - and it's actually more applicable to information evaluation and use for academic purposes than it looks like at first glance - is "Do you want to be more like these people in the ways they are like each other?" If not, you maybe want to look more closely at the details, or move on.

(In other words: communities produce specific community cultures. These affect what gets talked about, how, and how people react to those conversations. Academia, or journalism, or [pick your online space] is just another example of a community culture.)

It is not at all uncommon for recon groups to develop some sort of in-group jargon or weird interpretations of academic material and then take that stuff to scholars - who wind up staring at them blankly because that is stuff someone made up, usually based on a weird read of something or a wild extrapolation, and its connection to the academic understanding of the topic is basically nonexistent.

And also, that that wasn't what they were focusing on at all, and/or they're trying to apply academic techniques to something where that doesn't work, or expecting all disciplines to work the same, or other issues.

(I've been listening to a lot of the Archeological Fantasies podcast recently, which takes on pseudoarchaeology, and basically every episode has half a dozen examples of this.)

If someone's selling something else, likewise, check the thing.  Maybe you can't get a family tree of what all fornicated to make the thing happen but that's good to know.

And again, sometimes this is not necessarily public info, or at least not right out there.

(I hit this one fairly regularly when talking to people about my background: I am part of a tiny tradition, and my websites are basically the only online presence that mentions specifics right now. I have a really brief summary of some of where we come from on the group-and-training site, but basically say "If you want more details, I'm glad to do that in a one-on-one conversation.

That's not because it's private, per se, but because where I need to start that conversation is very different for someone who has familiarity with the Minnesota Pagan community from about 1995-2010, than for someone brand new to witchy tradition lineage, than for someone who's familiar with one or more of the commonly referenced trads. And on the whole 'glad to talk about this in person' seemed more useful than a Choose-Your-Own-Adventure level set of links and explanations. But the point remains you can't find a really clear description online on your own.)

How firmly and absolutely are things presented?  How flexible is the tradition?  Does it take into account new developments in unrelated fields which nonetheless have repercussions?  When there are firmly held positions - often on things like purity, appropriate or inappropriate offerings, and so on - how does the group deal with people who violate them?

This, so much this. Also understanding some of the benefits of experts and what they bring to a discussion (by which I mean people who have invested a substantial amount of time in a thing, and work with it regularly, maintaining and expanding their skills and knowledge in that area.) My blog post for Pagan Bloggers sometime this month is going to be about experts.

[Jenett]

I personally make a maybe persnickety distinction between "evidence" and "proof".

This is a distinction that works for some people, but isn't one that makes a lot of sense to me (perhaps because I spend most of my time doing stuff in the humanities.)

What I usually look at are facts (this thing happened on X date, we are really sure because of multiple sources) and then what information we have beyond that. Then it comes down to weighing the evidence, and figuring out which bits are most reliable.

Here's an example: one of my projects at work is gathering information about our student files, and what's in them. In the process we're finding all sorts of weird quirks, in part because parts of the files were retyped sometime in the first half of the 20th century.

So, we have "This piece of paper has X date" and "This one has Y date" and we can look at other evidence to suggest one is a typo (because the student files are roughly chronologically numbered, one of those dates usually fits the sequence). Or we look at which pieces would have referred to each other, and which come from independent sources, and figure out what's likely. 

What's in the file is a fact, but how useful it is is, and what it tells us, those are not 'proof', but they're evidence in building up a larger picture. We know that sometimes information in the files is inaccurate for various reasons (or that the family stories that got passed down are different than the information we have), and navigating that ambiguity or uncertainty is part of historical research.

The other part is that people's experiences of their world matter, but they're often very hard to capture - even letters or writing capture only what was set down on paper at a particular point, and their opinions may have changed, or they may view something differently with more experience or different experiences of related things. And I think 'proof' is... pretty limited, in this case. At best, what you can say is "At this point in time, they wrote this thing down."

It's a process that works for religious experience, too, but I think trying to force it into too rigid an academic or investigative framework damages the experience (and changes your outcome.) Self-awareness and pauses for review are good! But you don't stop experiencing the world just because you want to do evaluation.

[Jenett]

On this, as someone who studies history, often with biases sources, I find myself thinking of ad hominem. I know that ad hominem is generally regarded as a fallacy, attacking the source of an argument, not the argument. But as a historian, knowing who makes a claim, and what their motives are, is an entirely logical thing to do. Its just obvious. People have agendas, biases, and are sometimes ignorant. And I'm not the only person to have noticed this; I think a few philosophers have also pointed this out.

You're quite right. However, taking someone's experience with a subject (or past commentary) isn't ad hominem - the term gets used too broadly, to my way of thinking.

I just found an article from Scientific American that breaks down some different uses, and how they actually apply, with specific examples. As they point out, sometimes someone's background matters a lot (Are they a politician who ran on an anti-corruption platform, and there's corrupt behaviour coming out? That's a totally legit thing to talk about.)

What's ad hominem is dismissing someone's material because of who they are, or attacking the information because of who is providing it, when that's not actually relevant to evaluation of the information.

If their information is bad, you should have other ways of doing that, among other reasons to avoid it, ad hominem attacks are just lazy. Again, we deserve better evaluation of stuff than the lazy methods.

[Jenett]

What would you say are the best independent sources to keep up to date on latest American news?

America is a huge country! Both in terms of regions (there are tons of regional cultural differences) and by diversity of population in all sorts of ways. Because of that, I don't know that there's a single set of recommendations I'd make, but I'll tell you what I do.

First, I am pretty deliberate about making choices about what info is part of my routine: I don't have the TV or radio on routinely much, especially at home/as background noise. This is partly for magical/religious reasons, but that's probably a different thread! So I pick modes that are less invasive but still informative.

A couple of mainstream media sources considered reputable
My subscription choices at the moment are: 
- The Washington Post
- a National Public Radio station (we have two in Boston, I get regular emails from WBUR and dip into WGBH when there's a big story. This helps me get the local/regional stories of interest.)
- The Guardian (not a US paper, but they're doing good US in-depth coverage, and I like getting a non-US perspective.)

All three of these will cheerfully send me an email several times a day with the most important stories, and I'll go poke around from there when it's a useful point in my day.

I occasionally contemplate adding the Boston Globe (my regional paper with national reach) but their subscriptions are a bit more than I want to pay for the amount of meaningfully new content I'd get. The New York Times is another classic choice for national paper, and a couple of others also do good coverage in general.

(My views on this are pretty classic East Coast Establishment, just to be clear about biases. On the other hand the East Coast Establishment runs a lot of things still, so knowing what they're up to is handy.)

If you're interested in learning more about a place in specific, an NPR station is often a good call: if you can deal with podcast or streaming radio, many of them have a regionally produced longer conversation show on weekdays.

That can be a great look at that area's issues: our local ones bring in the mayor of Boston once a month, the governor, various local officials about projects or news stories, plus a mix of things like authors or experts in a topic. (They're not nationally funded - they get some government money but it's mostly listener/viewer donations and minimal advertising.)

Methods for exploring a wide range of other topics I wouldn't seek out
I mostly rely on skimming through Metafilterand reading posts of interest, but the politics threads there are a good source of detailed links and info. (It's a $5 one time cost to create an account, but you can read as much as you like without one.)

For cultural differences stuff, you might find the Ask Metafilter portion of the site (which is people asking personal questions like "How do I do this thing" or "How do I navigate this thing?") helpful to skim through, and read the ones that seem puzzling.

The blogs Ask A Manager (work advice) and Captain Awkward (personal advice) also get into differences of approach and talking through some common and less common patterns. Both have active comment sections (skim as needed), and a number of US posters, but also people from other places.

I also read most of the stuff that comes through Longform.org (curated selection of long-form journalism articles) for my burst of 'information I might not seek out'

Beyond that, I also do some deliberate dipping into alternate view points, but I make it a deliberate thing, not a part of my daily routine, and choose to do it once or twice a week, on average. Enough to know what's coming through, but not so much it takes over and I'm constantly in a state of reactive anxiety and distress. Those are not good ways to learn or do the stuff I need to do that does actively improve the world.

Podcasts
I listen to a bunch, some just for fun, but there are four worth mentioning for your purposes:

On the Media : two experienced co-hosts look at current media stories (mostly US focused, but not always) with analysis, commentary, and digging deeper into what's going on.

Backstory : A US History focused podcast. They changed format a bit this last February, but there are four co-hosts (of whom three will be on any given show, and one of whom is now a woman, despite the subtitle on the site) who are specialists in different eras of US history, and the shows mostly focus on a theme or idea (censorship, statistics, shame and reputation, utopias, etc.) and they've started doing occasional more current-events focused ones.

 The normal shows have a series of 2-4 segments with some discussion in between, and they tend to be really good, and they're paying a good amount of attention to diverse perspectives across a bunch of areas. (This is one of my regular listen-while-swimming ones, and I'm currently in the middle of an episode about local history with four very different takes on it.. As an example.)

This American Life:  Mostly worth mentioning here because while it's often a particular take on American Life, as it were, there are a lot of individual shows that are great places to start with issues or topics. A good source of personal experience stories with a thing. Huge backlog, so dipping in and out is a thing.

Note to Self : Show about technology and how we use it, but with a particularly good focus on a broad variety of people who use tech, and how, and lots of individual example cases.

Why the broad range of stuff?
For me, it's about rooting new stuff into stuff I already know: the more I have some general understanding of a thing, the easier it is to make connections, find new things, get up to speed on something, etc. I don't have to be obsessive about it, but at least skimming the above and reading things in more detail that make me go "I know nothing about that" or "I want more details about that." works out pretty well.

(I do pay special attention to "I know nothing about that" and "I don't know that I want to know things about that." There are some subjects I largely ignore - sports, the lifestyle choices of the rit

[Yei]

You're quite right. However, taking someone's experience with a subject (or past commentary) isn't ad hominem - the term gets used too broadly, to my way of thinking.

I just found an article from Scientific American that breaks down some different uses, and how they actually apply, with specific examples. As they point out, sometimes someone's background matters a lot (Are they a politician who ran on an anti-corruption platform, and there's corrupt behaviour coming out? That's a totally legit thing to talk about.)

What's ad hominem is dismissing someone's material because of who they are, or attacking the information because of who is providing it, when that's not actually relevant to evaluation of the information.

If their information is bad, you should have other ways of doing that, among other reasons to avoid it, ad hominem attacks are just lazy. Again, we deserve better evaluation of stuff than the lazy methods.

Excellent. That's a good article. And a much better explanation of ad hominem.