Tagging Open Call /​ Discussion Thread

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread

Contents

How do I help tag?

Option 1: Dive right in!

Though we have some guidelines, it’s totally great to just go to post pages and start tagging them with what feels like the right tags. You can even create yet-to-exists tags without worrying too much. Better you dive in and we do some clean-up than you don’t get started because it’s too much work to get started.

Option 2: Some helpful hints

We’ve worked to prepare answers to all the questions we’ve encountered so far in the Tagging FAQ. It covers and when and when not to tag, guidelines for creating tags, and some notes on tag voting. Ultimately, we’ll aim to fix up all tags to be in-line with the style guide described there. Feel free to comment there with any questions not yet covered. Good places to start It’s a good idea to start by becoming familiar with LessWrong’s existing tags. You can see them on the new Concepts page. Then are a couple of tagging strategies: Tag-First

Growing a community

We’d like to build a small community around taggers – the people who maintain the ontology of LessWrong’s library ensuring that desired information can always be found. Soon we’ll have Discussions Pages for every tag, but **in the meantime, if you want to connect with others about tagging, please comment on this post.**If you have any questions whatsoever, please comment here, DM me (or the rest of the team), or email us at ruby@lesswrong.com or team@lesswrong.com Thanks!

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=GQj5j5coLsTkcbogi

*Note: *I currently lean towards changing the Progress Bar metric from "tagged posts over 25 karma" to 35 or 40 karma. The original reason we went with 25 karma was an awkward compromise due to the LW2.0 karma inflation – old upvotes were only worth 1 point, now regular upvotes are worth 2 for most longtime users, and strong upvotes mean the average is more like 3-4. We haven’t gotten around to re-running the old vote history with the new vote-weighting, and that means that old (often great) posts have much lower karma than modern posts. We plan to bring the old and new votes in sync someday, but didn’t have time to do it this week. For modern posts, the threshold I’d have preferred to set was ~50 karma. This was roughly the equivalent of 25 back-in-the-day (hence the original metric). But I don’t really want to make people feel obligated to tag a bunch of mediocre modern posts – I’d rather taggers start shifting their efforts towards improving tag descriptions (turn stubs into full fledged A or B tier tags), and thinking about how the tag ontology fits together (i.e. are some tags duplicates? which tags are related?) My current guess is we should set the threshold to 40, and then I’m just going to strong upvote a bunch of older posts that deserve it to bump them over the threshold. (Meanwhile, to all the users who have doing doing tons of tagging: thanks!)

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=Tr8ajCR9DuERjpuWt

Update: I didn’t end up doing this, in part because it felt kinda lame to change the Progress Bar after the fact, partly because I realized it was relatively easy to at least get everything tagged with one of the core tags. (it’s still nicer if posts can get More Specific Tags) I would like to get the Progress Bar completed this week. Would anyone be interested in a zoom call coworking session where we set aside an afternoon or evening (maybe tomorrow if people are around) and then just… try to marathon the last chunk of tags? (note that this isn’t the last of the tagging work – the next chunks will probably be focusing more on "cleaning up the existing tag ontology, and making sure all the processed/​approved tags get a one-paragraph description at least, and/​or improving the text quality)

Comment

I would join such a zoom call. I have also been feeling the call of the progress bar’s final phase.

Comment

And my axe mouse!

Comment

And my touchpad!

Update: I will do this tomorrow at 1pm Pacific Time if that works for others. (I will post a Zoom link when the time comes)

Here is the Zoom link for the Progress Bar Hackathon

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=LbREQ39pyYJrznw3R

Someone mentioned that they thought the Concepts /​ Tag Portal was really nifty and they only just got round to looking at it, and that they thought it was motivating for tagging. I probably should have included a screenshot in the text (just added), but here’s a comment with a larger one:www.lesswrong.com/​tags/​all The new Concepts page

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=oGmD7RRQP4jguzCAd

PSA: Voting on relevance is an important, underserved, and easy to contribute to area of the tagging system. One person can create a tag, make a good description, and find a bunch of posts that fit it, but it takes multiple people’s votes to create a decent ordering of posts from most to least relevant. Which posts are listed first will be an important part of the user experience. This will be especially important for the more crowded tags, like the core tags, history, math, science, statistics, ai risk, and so forth. Contributing can be as easy as just going through the list and upvoting posts that you’ve read and think are a good fit for the tag. Edit: It would be nice to have a spreadsheet sorting tags by something like average relevance karma per post, to identify which tags most need votes.

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=SmEByQRDmoJur4MtF

Hey Multicore, I’d be interested in your thoughts on this alternate voting system I had proposed awhile ago, which (among other goals) aims to shift things such that it’s less necessary for multiple people to collaborate to vote on stuff. The team has gone back and forth on whether it’d be an improvement, and/​or whether it’s enough of an improvement to be worth the dev-cost to switch.

Comment

I agree with most of your analysis in the comments (many downsides to karma, multiple choice has some advantages intuition-wise and makes it easier for a single user to make an ordering), but I thought of a couple more points. My mind seems to only be coming up with downsides of the multiple choice system, which might be because I’m prone to rationalizing why the status quo is good.

  • Multiple choice has strategic voting implications too. If I think a 150 karma post and a 50 karma post are both "Top" relevance, but that the 50 karma post is better, I might rate the 150 karma post as "high" or lower.

  • Multiple choice makes it harder to see where in the ordering your vote would make a post end up. Additionally, your vote either has no immediate effect or moves the post around by a lot, so a fine-grained adjustment is impossible. That might not necessarily be bad though, if post karma mattering is desired.

  • If ordering is based only on the median vote, this makes it easy for a troll to vandalize a tag page even when the tagging system is mature. Just put the tag on a bunch of posts that don’t already have it and rate them all "Top". With karma, the post order is more stable once a lot of people have voted. (This is the double edge of making it easy for a single person to have a big impact.) However, these concerns balance out against the benefits you listed, so overall I don’t have a strong opinion on which is better.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=oLv5jkwrTtNBEaGrD

A way I can contribute to the site without having to come up with brilliant original ideas? Excellent!

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=4bqbX7H2YNnoyKZcS

This is probably a joke, but in my experience, explaining other people’s ideas is also a valued contribution if you explain it well and people are interested in the ideas.

Comment

I’m pleased to say that based on the bunch of tagging Multicore has been doing (we’ll make a tagging activity dashboard generally available soon), the above comment was entirely genuine.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=3N6kkn8HESmFpPJRw

The Progress Bar has been filled! Dubious honor to https://​​www.lesswrong.com/​​posts/​​XCsKmLw8L55ZyYjnK/​​think-before-you-speak-and-signal-it as the last post with 25+ karma to be tagged.

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=CAaheLpuyt7458f6p

I had the dubious honor to tag it! Thanks, everyone, for your work!

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=zooLbQuCak4BhmKrQ

Woop! It’s official.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=Mh4TK4vbodJhN7stQ

Making tags is fun!Here are some tags i made:Tagging Open Problems Summaries Prepping Meetups (topic) Category Theory (this one needs work because i know very little about category theory, but it seemed like a tag that should exist) Urban Planning /​ Design

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=kTPt7TXzgP4cG9Xsy

Hey! I think it’s cool that you created new tags. That being said, I do think that your description of category theory is not only a stub, but completely uninformative and dismissive about a mathematical field that has almost 70 years of work in it. I do think that explaining the controversy on the applicability of category theory is valuable, as we should question whether to use it for rationality and AI. But that should be a note at the end of the tag description, not the entire content of it. (Note that I didn’t change the tag description, because I don’t want to force a change if I’m the only one thinking that. Maybe the point is only to describe how the word in the tag is used in LW, in which case the current tag description might work.)

Comment

I agree with you, i struggled writing the description for it as i know very little about it, i just saw that it doesn’t exist yet and went ahead creating it (maybe it would have been better if i left it with no description?). so i say go ahead and edit it, you’ll surely do a better job than me :)

Comment

I wrote a first version of the new tag description, I might rewrite some parts of it later. ;)

Comment

Thanks, much better :D

+1 to be people creating tags they think are needed even if they’re not sure about the description.

Comment

Thanks :)In that case is it better to not write a description or write something knowing there’s a high chance it’ll be wrong (and leaving a note about it)?

Comment

My general philosophy is "better a description than no description".

I think it was the right call to write the tag descriptions that you did :) Yeah, I’d continue to write something knowing there’s a high change it’ll be wrong.

ahahaha that tag description is hilarious (no offesne Yoav, thanks for making the tag!). adamShimi, please feel very welcome to change that tag description.

Comment

Well at least i made someone laugh :D

As I’ve said a bit elsethread, I’m very in favor of 1) people creating tags they think should exist even if they don’t know the topic well, 2) other people jumping in improving them where they see possible. I hope talk pages will make it easy to discuss any changes made, but most of the time I expect they’ll just be all-round welcome improvements that don’t need debating.In short, if you think there’s an improvement to be made, go for it! Even if you’re not sure that anyone else agrees. Let them object after the fact.More philosophical, I think we generally want to define terms like category theory in their proper standard usage. If the LW usage is off, we probably want to correct that or at least note the divergence in any description. (I don’t remember if the S1/​S2 description, but it would be good if it noted the difference between academic meaning and how it gets used around here).If you end up writing an improved Category Theory description, I look forward to reading it.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=yCkrdhwZKRh72R2EQ

Thanks so much for diving in and doing so much! I’m so glad it’s fun. :)

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=jfNcWaYKfZiS4DB4X

Are there any thoughts on external links for tag wiki pages? I was looking at the social status tag, for example, and there are a few overcomingbias /​ ribbonfarm posts which I think would be useful but whether /​ how best to incorporate them isn’t clear to me

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=a4wjfAt29u7LgbsC8

How about creating link posts for them?

Comment

I think this is pretty reasonable. (Ideally the link posts also come with some summaries. In some cases it makes sense to ping the author about fully crossposting it)

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=vnEsc4Lrv38eYQMEC

My thought is they’re great! I’ve generally hoped the descriptions-texts would contain more of them. I explicitly listed them as a factor in meriting a A-Class or B-Class tag-grade. I think a good way to do this is to have an "External Resources" section as part of the tag description that lists the external resources (with or without some context on them).

Comment

I wondered about that too since, for example, open problems in group rationality isn’t on LessWrong, but is probably the main article on Group Rationality.

Comment

I vaguely remembered there was something like that and was surprised it wasn’t on the tag, so seems pretty good if it’s there in some form.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=amr2RWHvrmGabKMvF

Quick note that there is now a Tagging Activity page available to everyone. You can see users who have tagged posts or upvoted/​downvoted their relevance. (Corresponding reminder that tag activity isn’t private)

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=xnTpiyj7thsESyk5P

Thanks for the project, and the FAQ. I shall contribute. Is there a way to retrieve the old tags from LW 1.0? I remember they were used to index Open Threads, for instance. I can’t remember the details but that could be a good way to jumpstart some tags.

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=Wk6WGWGHjkwDXyL4Z

Awesome! Thanks! Don’t hesitate hesitate to ask for any help making the process easier. Re: LW1.0 tags—good question. I believe they they were implemented via the old LW1.0 wiki and you can still view all those pages and their tagged posts. https://​​wiki.lesswrong.com/​​wiki/​​Special:AllPages We’ve been planning to probably import that content, we estimate about 100 high quality articles/​tags, but it’s been waiting behind other dev work that seemed higher priority, e.g., Talk Pages for tags. We’ve also got our eyes on the Arbital content, however that’s a lot more work since we don’t currently have all the features that Arbital uses.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=GJiTm3skdevE4KTJb

I’m not sure the Newsletters tag quite makes sense. Generally if I’m interested in one Newsletter, what I want is more of that newsletter’s topic (or, more of that Newsletter, specifically), rather than other random newsletters I suppose it’s still reasonably handy for finding a specific newsletter post that you half-remember.

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=PaNLAar2nPhf8CLTv

I can see the usefulness of the tag. i think a way to solve the problem is to add links in the description to various newsletters and sequences that collect newsletters.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=rKnz2pS8ACjTAmLD5

There now seems to be over a 100 Sequences (depending on how one counts). I think it would be good to have tags for Sequences, Especially since they’re not easily searchable through a search engine (or not that I’m aware of, at least).

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=c6DvvMTvxZp68S2X9

PSA: You can now add and edit tags for a post from the drop down menu to the right of it (in places like the front page, tag pages, user profiles, etc.). This makes it much easier to go through a big list of posts and adding relevant tags. you can go to one of the lists that are linked in the progress bar in the main page and start ploughing through the posts. Thanks for the LW team for enabling this feature :)

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=gRh3HqDzH8JA8P86a

You’re welcome! Thank Raemon for that one. Now that you can tag without opening a post, I’ll say it’s better to only tag if you’ve read it or the tag is very obvious. Some posts can misleading /​ not clear from title and hover-over alone.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=uoiYJ8hkzgTXNWfse

Just a small note, i think it would be nicer if the tag editing popup was higher, so when you hover over a tag you’re able to see the description popup for it.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=Z4fP3cr3JPaiYFcsw

When I click "Add Tag", this is what I see:

Then I clicked to show more, because I know there are a lot more tags and want to make sure that if I tag a post it has all of the proper tags (because if I don’t it’ll be marked as tagged and it’s likely that no one will return to it to add the proper tags):

But this view isn’t organized well like the concepts portal is (below), so I felt the need to skim through each individual tag, which took a long time. Seems like it’d be a good idea to organize the above view to look more like the below view.

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=ToEfyXnQhWo6ERcC3

Agreed. It’s just actually a bit annoying to get it to work right, due to how the algolia search function works.

Comment

I think that adds to the reasons to perhaps replace Algolia.

Comment

I don’t think this issue would go away with replacing algolia. The problem is a sort of generic "algolia is faster than making a database query, but that speed comes with some default settings that require re-wiring, which I’m sure is possible but requires some upfront costs that aren’t part of my usual workflow. I think any search engine would come with the same issue."

Comment

(Don’t know how much the rest of LW wants to hear our internal dev discussions, but) it’s also things like Algolia, at least on our current plan, doesn’t have personalization, e.g. to recommend tags a person previously used or tags we algorithmically guess would apply this post but would want a human you check. Mostly going off Oli saying leaving Algolia is would be the way forward here. You might be right that no other solution will be better for this particular thing.

Comment

I am pretty confused what any of this has to do with Algolia. The primary problem to me appears to be that we don’t actually have a large fraction of the tags categorized in the tag hierarchy displayed on the All Tags page. We could show you a copy of the tag page table, but that would omit a lot of new tags, and also probably not be dense enough. We could develop some custom UI for that menu to group them, but that’s mostly a bunch of work (and doesn’t have super much to do with Algolia). The site search will probably always have somewhat different constraints than normal database operations (in particular if we want to stay within the autocomplete paradigm), so I don’t think anything about this would get easier if we switch away from Algolia (things like this are actually a domain where Algolia is pretty great).

Comment

I stand corrected and I hope Algolia is accepts my apologies for the slight. The actual table I don’t think is much a possibility, if desirable at all, but structured things are good. The alternative is just ordered things, if we can accurately predict which things are likely.

Comment

Yeah, ok. I do think personalization is blocked on Algolia, and I didn’t really think about this as a potential solution to this (but it totally is). So yeah, maybe slighting Algolia was the right call.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=w5kmHbGhDsSGtWosC

Just commenting to say it’s pretty cool to see the bar filling up and the number of tagged posts growing up. Thanks to all the taggers!

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=kNvcxN4LsYrKCECGY

Woop! We haven’t yet built any ways to recognize or reward the taggers, and I’d really like to. Any suggestions for how to do that? Also, I don’t want to shower a lot of attention on someone if they don’t want it. We did say that tagging activity would be public, but I don’t think that’s salient yet. If you’ve tagged a lot and don’t want a shout-out, please DM me before we get around to doing something proper. One short-term idea: if you’ve been tagging a lot, you can comment here and I can tell you how many tags you’ve applied to date. Others could then upvote your comment if they wanted to, to say thanks. Also general intangible prestige.

Comment

We haven’t yet built any ways to recognize or reward the taggers, and I’d really like to. Any suggestions for how to do that?

Publish a book of the best instances of people applying tags to posts in 2020.

Karma is nice. Maybe simply an appreciation post at some point, which could still not name people. Just let them know that they are appreciated. I don’t know if that’s possible, but another option might be some sort of "rank" or "badge" for top taggers. That being said, one might ask why have ranks only for this specific case, and not in general.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=uSaTEYxb4wdZLXiLL

The progress bar in the main page is super cool. I’m curious, how is it being calculated?Edit: oh, when you hover over it it says "X out of 4735 posts have been tagged (filtered for 25+ karma)" So i guess that’s how

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=TDEKomFBtXfM33YkY

Is it just me or is the number of posts tagged that’s shown on hover is going down with time?

Comment

Ahh, hmm. That is embarrassing. Hmm, I wish I had a better excuse for this. Hmm… I mean, look over there a three-headed monkey! (Will be fixed within the hour) Edit: And it’s fixed. Sorry about that!

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=NSRu7qbrkuNYjpyRE

I notice that when you try to tag a post but its relevance was in the negatives and your vote doesn’t bring it above zero, the site doesn’t give any feedback. It looks like there was a bug, or your connection messed up and didn’t submit the tag properly. This is somewhat mitigated now by being able to look at the tag voting page to confirm that you vote went through, but a lot of people won’t know to do that.

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=fTrEMFw6nrNj6Am4u

Yeah I also ran into that today. Will work on a fix of some sort soon.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=PFd74YRKMeCW42Htb

I’m not sure how to distinguish between the Empiricism tag and the Philosophy of Science tag. Anyone have a strong sense of what goes it one vs the other?

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=mAge7dkFiboSgv8nN

My sense is that Empiricism is specifically about experimentation and making beliefs pay rent in anticipated experiences, while a lot of the posts in Science are about academia and the social institutions of science. There’s some overlap between them. Additionally, Empiricism overlaps with Anticipated Experiences and Science overlaps with Replication Crisis. Empiricism was added more recently. I went back and tagged some of the posts in Science as Empiricism.

Comment

I agree with this; I wouldn’t expect Empiricism to have posts like e.g. Is Science Slowing Down? or Ed Boyden on the State of Science.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=pXKqYMHf7AKK3YG5S

I guess slightly new question: is Empiricism a strict subset of Philosophy of Science? Is there an example of an Empiricism post that isn’t in Philosophy of Science?

Comment

Maybe the "practical" side of empiricism.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=BkCRRGSZ2X8ie53vd

Hey, i added Empiricism. It seemed fitting as it is a frequent topic on LessWrong (surprise surprise :), it’s more specific than Philosophy of science (which currently has 63 posts), and is also named as one of the virtues of rationality.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=SPQq8bzurfcmgYehw

Tagging meta question: There’s a Calibration (Probability) tag. How important is it to keep that distinct from other forms of calibration (i.e. if you think some parameter will be within particular bounds, those bounds tend to be correct). Prompted by this post on time calibration. I suppose that all calibration is implicit probability calibration (i.e. if I think something will take between 15 and 45 minutes, I’m sort of implicitly claiming it has a high probability of being so, even if I didn’t concretely decide it was my 90% confidence interval). But, if all calibration can be reformulated as probability, do we need the "(probability)" disambiguation?

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=gc8c2hcuy8dsZn85u

Not all calibration is probability calibration, e.g., calibrating my scales or voltmeter, but as you suggest, calibration discussion on LessWrong is effectively calibration about credences/​probabilities. Not worth keeping finer gradations distinct. But I think the disambiguation is good because it explains the tag to someone new on LessWrong and doesn’t rely on your already knowing the content, so I’m in favor of the disambiguation. The way we use calibration is a our own jargon, so good to explain a bit what we mean just in the title.

Comment

I can see the case for that, but FYI it just made me go make this meta comment rather than intuitively classifying a calibration post. I think it might be fine to have the disambiguation live in the text rather than the title.

Comment

*Epistemic status: I’m generally pro disambiguations in parentheses, I was the one who advocated we borrow the practice from Wikipedia. I’m really not sure in the case between Calibration and Calibration (Probability), so I’m just trying to think through this with others. * The tag description is this:

Do the events that you give a 70% probability in advance, actually end up happening 70% of the time? It’s pretty brief, I’m guessing that didn’t clarify enough. I actually suspect here that if the description had been like the following, clarification wouldn’t have been necessary. Raemon? Someone is probability or credence calibrated if the things they predict with 70% chance of happening in fact occur 70% of the time. Importantly, calibration is *not the same as accuracy. *Calibration is about accurately assessing how good your predictions are, not making good predictions. Person A, whose predictions are marginally better than chance, e.g. 60% of them come true, and who knows that, is well-calibrated. In contrast, Person B, whose predictions are 90% accurate, yet thinks they are 99% accurate, is more accurate than Person A while being less well calibrated. Knowing how good your predictions are is a key rationalist skill. Among other things, being calibrated lets you make good bets [Link to Betting tag]/​make good decisions [link Planning & Decision-making tag], communicate information helpfully to others if they know you to be well-calibrated [link to Group Rationality], and helps prioritize which information is worth acquring [link to VoI tag]. Note that calibration applies to all expressions of quantified confidence in beliefs/​predictions [reference: Anticipate Experiences]. For example, calibration applies to whether a person’s 95% confidence intervals capture things 95% of the time. Or if their 80% chance of completion estimates are met 80% of the time (not much more or less). Trivially, odds ratio placed on things are convertible to probabilities. See also: prediction & forecasting I think this makes it much clearer that "tends to be correct" is always a quantitive/​probabilistic statement beneath the hood.Hmm, what this makes me think is really it’s about calibrating *credences, which isn’t standard jargon anyway. So maybe just plain "Calibration" is better than "Probability". *Or maybe Calibration (belief strength)? My main thought now is the issue wasn’t the name so much as lack of good explanation for the topic. [No complaint against the tag creator – it was a good tag to make.] I’d kind of like to have a big tag description writing push sometime after the plain "tag at all" push since so far few tags have good explanations. But we’d have to decide we’re definitely moving more in this wiki-ish direction. Yeah, I’d propose *Calibration *or Calibration (belief-strength). I dislike Probability Calibration because I dislike leading adjectives/​modifiers and prefer the main thing to be the first word in the noun phrase (some languages like Hebrew do this). I expect people to be looking for the core thing, e.g. Relationships, "R", and if you put modifiers in front, e.g. "Business", "Personal/​Interpersonal", "Romantic", "Conceptual", you then require someone to guess which modifier you used, and also split up Relationship tags from being adjacent in an alphabetical list.I think having as much in the title as possible is better than in the text, just because even triggering the hover-over is 10-20x costly than just skimming all the titles, and also doesn’t work on mobile where are no hover-overs. I think if someone’s looking over the tags list and sees "Calibration (belief-strength)" they have *much *more of an idea of what tag is about than just *Calibration *which is pretty opaque to an outsider.

Comment

I dislike Probability Calibration because I dislike leading adjectives/​modifiers and prefer the main thing to be the first word in the noun phrase (some languages like Hebrew do this). I expect people to be looking for the core thing, e.g. Relationships, "R", and if you put modifiers in front, e.g. "Business", "Personal/​Interpersonal", "Romantic", "Conceptual", you then require someone to guess which modifier you used, and also split up Relationship tags from being adjacent in an alphabetical list.

There is that, but at the same time, you probably wouldn’t want tags like Experiences (Anticipated) or Induction (Solomonoff). I don’t have any principled argument for this, but to me "Probability Calibration" feels more like one of those examples. It being put alphabetically close to "Probability" may also be good.

(I also keep feeling confused by Relationships (Interpersonal) each time I see it, though that’s probably in part because there’s no other ‘Relationships’, so I just think ‘well what other relationships could you even mean’ and then don’t find another Relationships that it would be contrasted with.)

Comment

That’s a fair point to raise. I think it’s more work to give an explicit theory for why those are different. Something like those are both technical terms/​jargon and they don’t belong to a broader class of things that we might be discussing. In a world where there were three types of induction we discussed, might then go for Induction (Solomonoff). Also that they’re the actual phrase people use. I don’t think I can remember anyone saying "probability calibration" ever.With Relationships (Interpersonal), I think it makes sense because to me, the default way to read "relationships" is specifically romantic relationships. Like if your friend says "I’m reading a book about relationships", what do you assume? To make it clear the tag also covers friendship, family, and work relationships I think actually does require some disambiguation even if the site doesn’t have any other relationship tags right now.

Comment

So perhaps we should also have Relationships (Romance)? :)

Comment

Indeed. <3

I went ahead and edited the tag to have your description

Comment

Many thanks!!

"Probability Calibration" rather than "Calibration (Probability)" feels like a more natural name for the tag, while keeping the disambiguation.

Comment

See my comment in the other thread about my argument for modifiers after the main word. It’s not LW team consensus, just something I’ve been pushing for.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=EdtZ2Bywnxvha544e

Would be nice to be able to see tags by date created, or something to know what new tags are created, so we can take a look and think if there’s anything that needs to go there.

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=MtMvKvqjXvRMMja9J

That makes sense! That’s the kind of thing we should build into a proper tagging dashboard.This spreadsheet has tags sorted by last changed (edited or posts added) but I’ll need to add in an additional column in order to sort by creation date. I can get around to it soon, if it’s helpful.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=QqadFgJgtfKHqDL3s

I have tagged a few posts from the top of the spreadsheet but not too many because it caused me reading too many old posts... I have added the tag Habits; hope that makes sense. I’m not too clear about the taxonomy. How frequently is the spreadsheet updated? UPDATE: Every 5 Min according to the OP.

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=SrpF7KQr8SLyMQvR6

but not too many because it caused me reading too many old posts... Darn! A pernicious failure mode indeed.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=4mP7CZXzLYESmPyLK

Thanks! Yesterday I was looking at the Martial Art of Rationality and trying to figure out what to tag it, and kinda gave up. I believe you created the tag for that? (I think it was the first tagged post). Kudos, it was a good solution I think.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=btHoDNYJd9ZvQ985c

We’d like to build a small community around taggers – the people who maintain the ontology of LessWrong’s library ensuring that desired information can always be found. Maybe this is a dumb question but, is this actually needed?Can we get what we want with people just randomly adding tags when they notice? Do we need to have people specializing on this? I’d expect that a bunch of work would be needed up front to get the tag system into a good state, but I’d think most of that work has been done already (by the LW team, and others). And then going forward I’d expect much less work to be required. Am I missing something?

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=JGa73rnfNYzuGJYrA

I think tagging is actually pretty hard. Like, by default you get a ton of synonyms of the same concepts, and there aren’t good redirects, and the tags don’t have good descriptions, and there is lots of ambiguity, and when someone creates a new tag old posts don’t reliably get tagged. Our tagging system is also more similar to being a wiki, and in-general my research into wikis suggests that basically all functional ones are maintained by a relatively small group of highly dedicated editors, and that it generally doesn’t work to just have everyone randomly edit and add things.

Comment

That’s helpful context. Makes sense, thanks!

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=pRtrHt5wBQW5z2fPS

Alternatively, we have an > automatically updating spreadsheet (every 5 min) that tracks the tags on the most viewed posts according to our data and their current tags. Note that this spreadsheet seems to open by default to the "sorted by karma" tab, and you have to manually switch to the "sorted by view rank" tab (spent half a minute wondering whether the link was incorrect before happening to look at the tab list). (Also: Oh wow, our most viewed post in one with four karma?)

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=kZ4qhQLYZcNNLK7Sr

Note: This is most viewed for the last 30 days I think. That post in particular tends to show up at the top every few months whenever you get closer to U.S. College application periods. And yeah, sometimes the posts that get a ton of views really aren’t very good. For a while one of our most viewed post was one at −4 karma called "The Effects of Religion [Draft]". It‘s been one of the reasons why I’ve been hesitant to include views as an easily accessible metric on the site, because I know how frequently it diverges from quality.

Comment

*Last 90 days

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=7eeXPT6pfm7qWqgSX

In https://​​www.lesswrong.com/​​tag/​​intuition?showPostCount=true&useTagName=true the description seemed to be copied from Wikipedia but the content is not licensed in a way that’s compatible with Wikipedia’s license.

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=XaCuxdumS94oHyrHa

I added the above notice about licensing requirements, which I think should make this fine.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=ovsAJDQ5pKDuPH4L6

Thanks for flagging this. I think we just have to add attribution? Will work on it.

Comment

It’s a share-alike license, so attribution alone is not enough.

Comment

Habryka (who has a bunch of copyright knowledge) thinks that appending the following is enough:

(From Wikipedia 1, 2, this section is licensed as CC BY-SA 3.0 Unported License* *to conform with the Wikipedia license requirements)

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=MWDt45XWAwsvtqvYB

It might be good to distinguish "content type" tags from "format type" tags. (Examples of the former are "AI risk" or "Introspection", examples of the latter are "dialogues", "open threads", "fiction").

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=TNwkKWt2FzPEdvrcA

lesswrong.com/​​tagvoting isn’t loading for me. All I get is the error " Error: TypeError: Cannot read property ‘isEvent’ of null". I’m using Chrome, if that’s relevant.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=RrRawf44hZjgfs4jw

I wanted to create a ‘Law and Legal Systems’ tag, but then i saw there’s a Government tag, currently only with 2 posts. should these tags be separate, or should i change the current tag to "Law, Legal Systems And Government"?

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=oFRyajsNDFfdPYeb2

They seem like different things to me.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=DNfMjzQKCdspDwPho

The tags page is occasionally suddenly replacing itself with the message "Error: TypeError: Cannot read property ‘_id’ of null", forcing me to reload the page. Has anyone else seen this? Edit: I also got the same error on the page for a post, when I added a tag, the server response was slow, and I tried to add it again.

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=NBoiQE5Arkz7NGMGy

Thanks! W e’re looking into it.

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=GA3GytLKL786esFPo

Happened to me too (though perhaps a slightly different error message, not sure)

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=doWcmfBHnD35iNBFW

It’s a good to start by becoming familiar with LessWrong’s existing tags. You can see them on the new Concepts page. In the text, Concepts shows up green, like it’s supposed to be a link, but it doesn’t go anywhere. Was it supposed to be a link?

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=RRWpw4pR4schBLYsw

Also, maybe it was supposed to be "It’s a good idea to start..."?

Comment

Thanks, fixed on both accounts!

Comment

The Concepts link is still not going anywhere for me :-/​. When I inspect on Chrome, it shows up like this:

Comment

Now actually fixed (there was a typo in the URL).

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=WJksRDtwfRkrBQy8s

Just realized when I tried to tag this post that there isn’t an Abstraction tag. Should there be one, or is there an equivalent tag?

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=qsokG3z5cvrQAHEkW

I thought to myself a while ago that we should definitely have an Abstraction tag, please go ahead and make it!

Comment

Done. It’s a stub, but I tagged all posts from the Abstraction sequence.

Comment

Awesome, thanks!

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=HjSeBNX9bG4XCy2HH

Diseased disciplines: the strange case of the inverted chart is an interesting case because tagging it correctly feels like a spoiler.

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=uvuB3tsdRdXyscbXj

After i read your comment i decided to read the post. i saw the tag programing*, but forgot about it midway and it didn’t spoil the surprise for me—still i think you’re probably right it would for other people. Anyway, i also added it to information cascades, as it seems relevant and doesn’t spoil it. *not sure how to do a spoiler tag

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=SHk7EAw4GkSu9kbmn

Would be nice to be able to add tags from the drop-down menu to the right of posts (in places like the homepage and users profiles). This would speed up the process by a lot.

Comment

https://www.lesswrong.com/posts/uqXQAWxLFW8WgShtk/tagging-open-call-discussion-thread?commentId=miAnKeM5mwmMBoeu3

I agree. Currently, there’s the "Edit Tags" button that gets you there, but we’ve been meaning to replace that with the "AddTag" button directly. Thanks for the feedback.

Comment

(quick note: the Edit Tags button is actually admin only. We can make that available to everyone soon though)

Comment

Ohh. I’m terribly sorry then, everyone. We’ll get that fixed.