It’s important to be aware of their opinions because they quickly become policy and rhetoric while Dems do damage control and fail to fix the underlying issues Reps exploit. In this case, having an instance where they directly contradict their own “sleepy Joe” narrative can help deconstruct the beliefs of family members who have fallen for it.

This is a sane and measured response to a terrorist attack /s Just do terrorism back 100-fold, I guess?

I think it’s moreso a matter of evolution. We know humanoid bodies can do anything we can do, so we start with that and make incremental improvements from there. We already do have plenty of other body shapes for robots (6-axis arms, SPOT, drones, etc) but none of them are general-purpose. Also, the robot shown in the article is pretty clearly not fully humanoid, it has weird insect legs probably because it’s easier to control and it doubles as a vertical lift.

Network effects. The “we” you’re referring to could only be like 100 million at most, the vast majority of people don’t have the technical know-how to switch, or to articulate exactly why they feel miserable every time they log in for their daily fix.

For what it’s worth I don’t think they’re proposing it will “solve” climate change - no single thing can. It’s millions of tiny (alleged) improvements like this which eventually add up to taking pressure off of the environment. I see this kind of attitude a lot with stuff like paper straws or biodegradable packaging, as if the idea of a small but meaningful step in the right direction is laughable. It’s fine to criticize them for the “improvement” actually being no better than the alternative, but I worry sometimes it comes across like any sort of improvement short of “solving” climate change isn’t worthwhile.

If we had access to the original model, we could give it the same seed and prompt and get the exact image back. Or, we could mandate techniques like statistical fingerprinting. Without the model though, it’s proven to be mathematically impossible the better models get in the coming years - and what do you do if they take a real image, compress it into an embedding, then reassemble it?

I respect your boldness to ask these questions, but I don’t feel like I can adequately answer them. I wrote a 6 paragraph essay but using GPT-4 as a sensitivity reader, I don’t think I can post it without some kind of miscommunication or unintentional hurt. Instead, I’ll answer the questions directly by presenting non-authoritative alternate viewpoints.

1. No idea, maybe someone else knows
2. That makes sense to me; I would think there would be a strong pressure to present fake content as real to avoid getting caught but they’re already in deep legal trouble anyway and I’m sure they get off to it too. It’s hard to know for sure because it’s so stigmatized that the data are both biased and sparse. Good luck getting anyone to volunteer that information
3. I consider pedophilia (ie the attraction) to be amoral but acting on it to be “evil”, ala noncon, gore, necrophilia, etc. That’s just from consistent application of my principles though, as I haven’t humanized them enough to care that pedophilia itself is illegal. I don’t think violent video games are quite comparable because humans normally abhor violence, so there’s a degree of separation, whereas CP is inherently attractive to them. More research is needed, if we as a society care enough to research it.
4. I don’t quite agree, rights are hard-won and easy-lost but we seem to gain them over time. Take trans rights to healthcare for example - first it wasn’t available to anyone, then it was available to everyone (trans or not), now we have reactionary denials of those rights, and soon we’ll get those rights for real, like what happened with gay rights. Also, I don’t see what rights are lost in arguing for the status quo that pedophilia remain criminalized? If MAPs are any indication, I’m not sure we’re ready for that tightrope, and there are at least a dozen marginalized groups I’d rather see get rights first. Unlike gay people for instance, being “in the closet” is a net societal good because there’s no valid way to present that publicly without harming children or eroding their protections.

The legality doesn’t matter, what matters is that the sites will be flooded and could be taken down if they aren’t able to moderate fast enough. The only long-term viable solution is image classification, but that’s a tall ask to make from scratch.

You’re right, apologies. Skimmed too hard

There’s a lot of papers which propose adding new tokens to elicit some behavior or another, though I haven’t seen them catch on for some reason. A new token would mean adding a new trainable static vector which would initially be something nonsensical, and you would want to retrain it on a comparably sized corpus. This is a bit speculative, but I think the introduction of a token totally orthogonal to the original (something like eg smell, which has no textual analog) would require compressing some of the dimensions to make room for that subspace, otherwise it would have a form of synesthesia, relating that token to the original neighboring subspaces. If it was just a new token still within the original domain though, you could get a good enough initial approximation by a linear combination of existing token embeddings - eg a monkey with a hat emoji comes out, you add tokens for monkey emoji + hat emoji, then finetune it.

Most extreme option, you could increase the embedding dimensionality so the original subspaces are unaffected and the new tokens can take up those new dimensions. This is extreme because it means resizing every matrix in the model, which even for smaller models would be many thousands of parameters, and the performance would tank until it got a lot more retraining.

(deleted original because I got token embeddings and the embedding dimensions mixed up, essentially assuming a new token would use the “extreme option”).

There’s a lot of papers which propose adding new tokens to elicit some behavior or another, though I haven’t seen them catch on for some reason. A new token would mean adding a new trainable static vector which would initially be something nonsensical, and you would want to retrain it on a comparably sized corpus. This is a bit speculative, but I think the introduction of a token totally orthogonal to the original (something like eg smell, which has no textual analog) would require compressing some of the dimensions to make room for that subspace, otherwise it would have a form of synesthesia, relating that token to the original neighboring subspaces. If it was just a new token still within the original domain though, you could get a good enough initial approximation by a linear combination of existing token embeddings - eg a monkey with a hat emoji comes out, you add tokens for monkey emoji + hat emoji, then finetune it.

Most extreme option, you could increase the embedding dimensionality so the original subspaces are unaffected and the new tokens can take up those new dimensions. This is extreme because it means resizing every matrix in the model, which even for smaller models would be many thousands of parameters, and the performance would tank until it got a lot more retraining.

LLMs are not expert systems, unless you characterize them as expert systems in language which is fair enough. My point is that they’re applicable to a wide variety of tasks which makes them general intelligences, as opposed to an expert system which by definition can only do a handful of tasks.

If you wanted to use an LLM as an expert system (I guess in the sense of an “expert” in that task, rather than a system which literally can’t do anything else), I would say they currently struggle with that. Bare foundation models don’t seem to have the sort of self-awareness or metacognitive capabilities that would be required to restrain them to their given task, and arguably never will because they necessarily can only “think” on one “level”, which is the predicted text. To get that sort of ability you need cognitive architectures, of which chatbot implementations like ChatGPT are a very simple version of. If you want to learn more about what I mean, the most promising idea I’ve seen is the ACE framework. Frameworks like this can allow the system to automatically look up an obscure disease based on the embedded distance to a particular query, so even if you give it a disease which only appears in the literature after its training cut-off date, it knows this disease exists (and is a likely candidate) by virtue of it appearing in its prompt. Something like “You are an expert in diseases yadda yadda. The symptoms of the patient are x y z. This reminds you of these diseases: X (symptoms 1), Y (symptoms 2), etc. What is your diagnosis?” Then you could feed the answer of this question to a critical prompting, and repeat until it reports no issues with the diagnosis. You can even make it “learn” by using LoRA, or keep notes it writes to itself.

As for poorer data distributions, the magic of large language models (before which we just had “language models”) is that we’ve found that the larger we make them, and the more (high quality) data we feed them, the more intelligent and general they become. For instance, training them on multiple languages other than English somehow allows them to make more robust generalizations even just within English. There are a few papers I can recall which talk about a “phase transition” which happens during training where beforehand, the model seems to be literally memorizing its corpus, and afterwards (to anthropomorphize a bit) it suddenly “gets” it and that memorization is compressed into generalized understanding. This is why LLMs are applicable to more than just what they’ve been taught - you can eg give them rules to follow within the conversation which they’ve never seen before, and they are able to maintain that higher-order abstraction because of that rich generalization. This is also a major reason open source models, particularly quantizations and distillations, are so successful; the models they’re based on did the hard work of extracting higher-order semantic/geometric relations, and now making the model smaller has minimal impact on performance.

LLMs are not chatbots, they’re models. ChatGPT/Claude/Bard are chatbots which use LLMs as part of their implementation. I would argue in favor of the article because, while they aren’t particularly intelligent, they are general-purpose and exhibit some level of intelligence and thus qualify as “general intelligence”. Compare this to the opposite, an expert system like a chess computer. You can’t even begin to ask a chess computer to explain what a SQL statement does, the question doesn’t even make sense. But LLMs are capable of being applied to virtually any task which can be transcribed. Even if they aren’t particularly good, compared to GPT-2 which read more like a markov chain they at least attempt to complete the task, and are often correct.

It’s actually false. You can’t teach them long-term, but within the course of a conversation they can be taught new rules and behaviors. There’s dozens if not hundreds of papers on it.

The fact that they can perform at all in essentially any task means they’re general intelligences. For comparison, the opposite of a general intelligence is an expert system, like a chess computer. You can’t even begin to ask a chess computer to classify the valence of a tweet, the question doesn’t make sense.

I think people (including myself until reading the article) have confused AGI to mean “as smart as humans” or even as “artificial person”, neither of which is actually true when you break down the term. What is general intelligence if not applicability to a broad range of tasks?

“You can use them for all kinds of tasks” - so would you say they’re generally intelligent? As in they aren’t an expert system?

Actually a really interesting article which makes me rethink my position somewhat. I guess I’ve unintentionally been promoting LLMs as AGI since GPT-3.5 - the problem is just with our definitions and how loose they are. People hear “AGI” and assume it would look and act like an AI in a movie, but if we break down the phrase, what is general intelligence if not applicability to most domains?

This very moment I’m working on a library for creating “semantic functions”, which lets you easily use an LLM almost like a semantic processor. You say await infer(f"List the names in this text: {text}") and it just does it. What most of the hype has ignored with LLMs is that they are not chatbots. They are causal autoregressive models of the joint probabilities of how language evolves over time, which is to say they can be used to build chatbots, but that’s the first and least interesting application.

So yeah, I guess it’s been AGI this whole time and I just didn’t realize it because they aren’t people, and I had assumed AGI implied personhood (which it doesn’t).

It feels kind of hopeless now that we’d ever get something that feels so “radical”, but I’d like to remind people that 80+ hour work weeks without overtime used to be the norm before unions got us the 40 hour work week. It feels inevitable and hopeless until the moment we get that breakthrough, then it becomes the new norm.

Good to note that this isn’t even hypothetical, it literally happened with cable. First it was ad-funded, then you paid to get rid of ads, then you paid exorbitant prices to get fed ads, and the final evolution was being required to pay $100+ for bundles including channels you’d never use to get at the one you would. It’s already happening to streaming services too, which have started to bundle.

I’m an AI nerd and yes, nowhere close. AI can write code snippets pretty well, and that’ll get better with time, but a huge part of software development is translating client demands into something sane and actionable. If a CEO of a 1-man billion dollar company asks his super-AI to “build the next Twitter”, that leaves so many questions on the table that the result will be completely unpredictable. Humans have preferences and experiences which can inform and fill in those implicit questions. They’re generally much better suited as tools and copilots than autonomous entities.

Now, there was a paper that instantiated a couple dozen LLMs and had them run a virtual software dev company together which got pretty good results, but I wouldn’t trust that without a lot more research. I’ve found individual LLMs with a given task tend to get tunnel vision, so they could easily get stuck in a loop trying the same wrong code or design repeatedly.

(I think this was the paper, reminiscent of the generative agent simulacra paper, but I also found this)

Huh, is this the start of a new post-platform era where we see such business models the way we now see cigarettes?

I wonder what effects this will have with all these antitrust suits happening right as AI is ramping up, but before any of them have got any real foothold. Maybe Alexa will never get a brain and instead AI assistants will be seeded by the breakups or startups untarnished by the end stages of their shareholders parasitizing value?

Nebula is so cheap I have a subscription even though I almost never use it. I would use it more if they had a better recommendation system, as it is now you almost have to search for a specific video you want or dig through piles of random videos you don’t care about.

Also worth considering with self-report that Gen Z may just be more open about their failings

😮 Where’s the link? I don’t use Discord but I like free /s

So how long until people start advocating for the nonhuman, nonsentient robot over actual PoC? “Silicon lives matter”?

This is victim blaming. He isn’t at fault for trusting a company to have the bare minimum of respect for his property and autonomy, the company is at fault for not actually having that respect. Whether the company is actually trustworthy is as immaterial as saying someone “deserved” to have their car stolen because they forgot to lock it.

You can criticize him for not being cautious in this low-trust environment, but don’t let it get to the point where the party actually at fault gets off without criticism.

They aren’t mutually exclusive, I have a few smart lights and I try to plug them into switched outlets so I can turn them off manually and also control digitally.

Can’t be a billionaire if you pass a certain threshold of self-awareness, it’s the rules.

Hm, yeah I think you’re right. I was wondering why it wasn’t sitting right in my head. Deflation encourages hoarding because the value of each unit keeps increasing so if you spend now instead of later you lose some amount of potential value. I don’t think it was meant to be a scam though. In this case I’d consider it ignorance of the knock-on effects later exploited rather than an explicit conspiracy from the get-go.

Bitcoin at least is inherently deflationary because there’s a fixed market cap of 21 million bitcoins. Once all of those are mined, all value from then on is some fraction of a fraction of one of those, thus they decrease in value over time. I should also note, I like Bitcoin as a proof of concept but don’t think it’s viable as a currency, and PoW isn’t viable as a consensus protocol (although it demonstrated that such consensus protocols are possible).

“Squandering” is a great description of what they’ve been used for. The only implementations I’ve seen thus far that seem genuinely useful are FileCoin and a few decentralized computing attempts like ICP (not Ethereum). I could see a potential niche use-case for NFTs to decentrally coordinate ownership of abstract properties like domain names, but speculative monkey jpegs ain’t it chief.

It’s fiat, I won’t argue it was ever going to be a good currency with built-in deflation, but that’s what it was originally meant to be. It’s long since become too volatile to be anything but a speculative asset, though. It does seem curious to me what that says about the actual distinction between legitimate currencies, stock options, and pyramid scheme buy-ins.

My hope is the fintech people will be gone by the time someone thinks of an eco-friendly consensus protocol that isn’t Proof of Stake (ie “the people with the most money have the most power”).

It’s an HTML-like language that defines the browser’s interface, you can use it to change the shapes, positions, colors, whatever of your toolbars and tabs. Also they do still have customization via userChrome.css and I think you can re-enable XUL if you dig enough? It does get mixed a bit with HTML-namespaced tags too.

I switched recently as my first baby steps to degoogle, particularly when I saw the writing on the wall with WEI. I was very pleasantly surprised by how customizable it is using XUL.

I have dozens of projects in varying levels of completion and maybe like 2 finished projects. Here’s my list, steal to your liking because I come up with ideas I want to see in the world, and clearly I’m not a great medium for that:

• Philotic - p2p network of Python servers based on a generalization of process forking. Every server runs the same file (global scope is initialization) but some id-based guard like an annotation lets them do different things. I designed this to act as a lower layer for eventually splitting an LLM across multiple computers, something very obnoxious to do manually but relatively easy to code.
• Servitor - Actually probably will continue working on this, it’s a library which makes it easy to use LLMs as “semantic functions”, effectively letting you describe a natural language task and execute it as if it were a normal function.
• TAO - Type Annotated Objects, more or less CBOR with some personal improvements. 0-0x7f are tiny ints, the first nibble is a type, and the second nibble is usually an embedded length prefix (or signal for a longer length prefix). Being nibble-based and having a dedicated DEBUG type makes it a lot easier to read in hexdumps, and gives twice as many type prefixes to work with. I rearranged the types a bit to be more sane than CBOR (which eg has separate types for negative and positive integers), and also added streaming and varint support.
• STM/Snow - Structured Text Markup (in-progress name, “Snow” is a bit too informal?), a text serialization format where all data is (un)quoted text, “tags” which are { followed by data or text: data pairs, then }, or “markup” which is [ followed by plaintext interspersed with tags and ending with ]. The mixed positional/named attribute model of tags makes its object model a generalization of XML using JSON-like syntax and I’ve found it to be very easy to implement parsing.
  • My “pie in the sky” dream is to completely overhaul HTML/CSS/JS for STM/simplified CSS/WASM, but that’s never going to happen 😞
• Munchy - IDL-like language for representing file formats as an executable schema. Eventual goal was for it to be powerful enough to parse textual formats like JSON, which tend to be more contextual. At some point I found a similar project using YAML to define the schemas, but not being a DSL made it more confusing IMO.
• RetroArch file - A common file format for RetroArch to combine ROMs, patches, cheats, saves, etc into one cohesive format. Never got far with this one.
• Binary MIME aka contype. I even wrote an RFC apparently? Adorable.
• LLM modification - A paper I wrote about a potential LLM modification replacing the FF layers with a shared vector database to decouple memorization objectives from semantic and syntactic objectives, resulting in smaller foundation models. Predictably no one cared and probably no one should care, but it might be an interesting experiment to implement.
  • Probably a more useful modification I haven’t seen yet would be to implement kv caching using a prefix tree rather than a per-request cache. That would make semantic functions a lot faster, since it could cache the prompt between requests and it would only have to process the data.
• Preference vectors - Simple stochastic updating of “preference” and “feature” vectors to transparently associate preferences with content. This would allow people to essentially build their own “The Algorithms”, since the update operation can be designed to create a linear space so you can eg request content close to “my preferences + my mood + heavy metal + randomness”, and share feature vectors on social media. I think when I tested it I made a weird modular space where d(0, 255) = 1, and it still worked. Stochastic updates work, even in a distributed context, because it’s a kind of “simulated annealing”.
• Wika - Simplified and standardized WikiText parsing (which is surprisingly not actually standardized and MediaWiki essentially defines it as “whatever our PHP parser can read”). Follow-up is naturally a wiki written in anything other than PHP.
• i2cec - ATtiny11 firmware for bridging the i2c and CEC lines of an HDMI cable so you can send remote control commands via your SMBus to an attached monitor (I accidentally got a TV instead of a normal computer monitor). Never got it to work quite right, timing was very tight.
• U413 - A unix terminal themed BBS forum with a looong history of makes and remakes and a community getting whittled down to a handful of people.

And finally then there’s my magnum opus, Espresso, my favorite project I keep coming back to time and time again and bikeshedding refining over many years. If anyone else takes it up I’d be ecstatic.

• Influences: TypeScript, Python, Lua, Rust, C/++, Julia
• Self-hosted prototype-based scripting language with its JIT written in itself (eventually)
• Emphasis on simple rules which build up arbitrary complexity, a pathological avoidance of repetition, conciseness, and near Lispian customizability. SMOL.
• ASCII lexing with unicode support deferred to other stages (compare to Lua, which treats > 0x7e as an identifier - I also treat <= 0x20 as whitespace).
• PDA tokenization (used to be FSA/regex but nested format-strings required more power).
• LR(1) parsing with concurrent bytecode emission (ala Lua), AST is built up until it can be converted to bytecode. The most extreme case is extensive destructuring assignment (Rust, Python, [P2392]) which shares a syntax with object literals, but can be treated as LR(1) by keeping track of a distinction between “lvalue” and “rvalue” compatible syntax.
• All types defined with proto[...T] Super Sub(...args) { ... }, eg proto class Animal { ... } and proto Animal Monkey { ... }
  • The higher-order types include class, enum, union, interface, struct, etc. Compare to [P0707]
  • Note that these kinds are objects, not keywords. They define how to convert the body to a type constructor and prototype chain(s).
  • It took a few months to figure out this is possible by maintaining separate prototype chains for classes and instances.
• Statements implicitly end, ; is an optional operator to explicitly disambiguate their ending.
  • after operator, x() after y() == (var t = x(); y(); t) - surprisingly useful for conciseness.
  • “Everything is an expression” - loops are generators, eg list comprehension works like [...for(var x in 10) x] == list(range(10))
• Operator overloads use the operator itself as the method name, eg proto class { +(rhs) { console.log("\{this} + \{rhs}"); } }
• Type annotations define compiler semantics: Type objects have a delegate() method which define how to represent the variable on the stack. Untyped variables use an inferred type or an implicit any, which wraps a dynamic object. This lets you create objects like int32 while still using the prototype semantics.
• Recently I thought the syntax itself can be partially dynamic by using static scoping and macros which hook into the compiler when they’re encountered, but I’ve tried similar things a few times and it tends to lead to failure. This would need something like C#'s unsafe compilation distinction to avoid catastrophic vulnerabilities.
• “Initialization is compilation” - When you run a file, executing the global scope (“initialization”) is treated as a stage of compilation, and the resulting module object is what is saved, not just its bytecode. Compare this to Python, which saves just the bytecode of the module.
• Lifetime semantics (ala Rust and friends) are still a WIP.
• Based on [P0709] and Rust’s try! semantics, exceptions are actually wrapped in a returned Result[T, E] type: try is an operator which unwraps the result and returns if it’s an error. Thus you get var value = try can_fail();. Using type object operator overloading, the Result type doesn’t need to be explicitly annotated because Result[T, E] == T | Result[T, E] == T | fail E.
  • fail keyword instead of throw/raise.
• Really want proper coroutines using PyPy’s continulet/stacklet abstraction. Also maybe delimited continuations as the implementation for panics.
• Structured process forking.
• GC based on ideas from this LuaJIT 2.0 GC document.

I could go on for hours with all of the stuff I’ve thought of for this language. If you want to know more, the README.md and ideas.md are usually the most authoritative, and specification.md is a very formal description of a subset of the stuff that is absolutely 100% decided (mostly syntax). I’ve written turing complete subsets of it before. First usable implementation to arrive by sometime in 2200 lmao. 🫠 I can also unpack the other projects if you want to know more.

Now I want to see his reaction when people start breaking out the guillotines because his ilk have made peaceful resolution impossible.

Gates is a weird figure - I don’t think he’s actually evil (nowadays), because evil implies malicious intent. Rather, he unintentionally commits evil actions (those which harm people and if committed with full understanding would be unambiguously evil) as a result of his own personal extremely warped values. The guy thinks patent law is the highest moral good, so the millions of people who don’t get vaccinated as a result are either forgotten about or a necessary sacrifice for the societal good of protecting patents (which he might justify as making these sorts of innovations possible in the first place - demonstrably incorrect, but good luck getting him to listen to anyone who might tell him that).

tl;dr you can’t be evil by human standards if you have alien blue and orange morality.

The key search term is “userChrome” (userChrome.css and userChrome.js) and XUL, which is the HTML-like language FF uses to define its chrome. “Chrome” is a term that predates Google’s browser, referring to the interface surrounding the displayed web content and Firefox still uses that internally.

Right now mine is pretty minimal, but there’s a lot you can change. Essentially, the interface is a kind of HTML page which can use the same features as normal HTML and can even run custom JavaScript. Also look into BetterFox for how to remove Mozilla’s own telemetry and bloat.

My userChrome.css for reference;

/* Move findbar to the top */
.browserContainer > findbar {
    -moz-box-ordinal-group:0 !important; /*  for 112 and older */
    order: -1 !important; /* for 113 and newer */
    border-top: none !important;
    border-bottom: 1px solid ThreeDShadow !important;
}

/* Remove "Open All In Tabs" button in bookmarks folders */
#PlacesToolbarItems .openintabs-menuitem,
#placesContext>menuitem[id="placesContext_openContainer:tabs"],
#placesContext>menuitem[id="placesContext_openContainer:tabs"]:not([hidden])+menuitem+#placesContext_openSeparator {
    visibility: collapse !important;
}

/* Tabs are attached on the bottom */
.tab-background {
  border-radius: var(--tab-border-radius) var(--tab-border-radius) 0 0 !important;
  margin-top: 1px !important;
  margin-bottom: 0 !important;
  padding-bottom: 31px !important;
}
.tabbrowser-tab[multiselected=true]:not([selected=true]) .tab-background {
  border-radius: var(--tab-border-radius) !important;
  margin-top: 2px !important;
  margin-bottom: 1px !important;
  padding-bottom: 29px !important;
}
.tabbrowser-tab[selected=true] .tab-background ,
.tabbrowser-tab[multiselected=true] .tab-background {
  background-color: var(--toolbar-bgcolor) !important;
  background-image: var(--toolbar-bgimage) !important;
}

I found this repo which is supposed to apply Chromium styles in line with Google’s Material Design guidelines.

Here’s an article I found with some simple tweaks.