Watch

Transcript

Disclaimer: The transcript that follows has been generated using artificial intelligence. We strive to be as accurate as possible, but minor errors and slightly off timestamps may be present.

Jacob Steeves (00:00):

Thanks for having me. Yeah, thanks for coming.

Jon Cole (00:02):

Yeah, awesome, really excited to be here.

Jacob Steeves (00:04):

And where are you from? You’re in South America right now?

Jon Cole (00:07):

I usually live in Peru, but right now I’m in Toronto where the majority of our developer team is. So I’m out here for the new years. We’re preparing to launch a new network. So that’s gonna take a lot of hard in-person work. Usually we’re pretty remote. The project actually started with us all just meeting online in a fully decentralized way. But for the holidays, we’ll all be in person here in Toronto. But yes, I usually live in South America where I call the mountains my home.

Jacob Steeves (00:42):

That’s awesome. And are you from the US?

Jon Cole (00:46):

No, I’m from Canada actually, from Vancouver originally.

Jacob Steeves (00:50):

Yeah. Oh, that’s awesome. Yeah, Toronto is a big hub for tech. I’m starting to realize that.

Jon Cole (00:53):

It’s a huge hub for AI, interestingly. A lot of the big names live here. You know, Geoffrey Hinton, Yann LeCun, the Vector Institute University of Toronto is here. There’s the, I forget the name of this AI. Canada recently invested like a hundred million dollars into one research lab here in Toronto. They really want to make it a hub. And then there’s obviously Montreal, which is very close. You have Yoshio Bengio.

(01:24):

You have Belayovsky. You have the Mila Lab, University of Montreal, where a lot of the ideas around things like neural networks and AlexNet. And the big name advances in machine learning over the last 15, 20 years have come out of these two cities. Interesting, I’m proud to be a Canadian and be so well-connected to the heart of AI, Geoffrey Hinton. But Geoffrey Hinton is actually not Canadian. He’s British, but many of them worked out of Canadian universities.

Jacob Steeves (01:57):

That’s awesome. And just before we get started here, we’re about to get started in one second. I’m just sending in the BitSensor Discord the updated Twitter space link. Oh, hoorah. Yes, sir. So let’s do our proper introduction. So guys, thanks for coming. This is a recorded space. So thanks everybody for listening in. If you couldn’t tune in, I am really excited to have BitSensor on. I think it’s one of the best projects currently out and one that offers a really unique perspective in the crypto space. And I’m really excited to deep dive today. And with me, I have one of the lead team members, Konst. And if you want to introduce yourself a little bit, and if you want to introduce yourself a little bit.

Jon Cole (02:38):

Sure, yeah, thanks. Thanks a lot, John. I really appreciate it and happy to hear that you’re interested in the project. We’ve been working on it for many years. I’ve been working on it for ages, it seems. And I’ve been there since basically the inception of the project, mining BitSensor and coding, some of the early developments of the project back when it was merely names and ideas.

(03:06):

I have a degree in mathematics and computer science. Formerly, I worked at Google on large scale computer science and machine learning. And previous to that, I was working building neural network chips. So from hardware to distributed machine learning to peer-to-peer version, decentralized machine learning that’s been my life since I can remember almost. Nice to see Scrimp here.

Jacob Steeves (03:32):

Hey, Scrimp. That’s awesome. I mean, that’s a really cool background. So I guess leading off that, so when did you get into the crypto space?

Jon Cole (03:40):

I got into the crypto. Hi, Dr. M, nice to see you here as well. When I was working building neural network chips for a subsidiary of like a contractor for DARPA, one of my mentors there introduced me to Bitcoin back in the day, taught me the gospel. And I got involved with a small underground Bitcoin space back in 2014 where they were teaching Bitcoin and they had some really interesting things. They had this like vending machine that you could pay in Bitcoin and beers would come out. That was later updated to lightning. And there was a lot of all sorts of weird hackers down there that were doge lovers and just weird zany people on the edge of civilization, you might say. A number of them went clinically insane years later. And these were the type of people that I remember, the cyberpunk types that were interested in Bitcoin back in 2014 and I fell in love with the space. And that’s how I got involved in cryptocurrency through that. I was always quite skeptical back in 14. I really wish I hadn’t been as skeptical with Bitcoin back in 2014, but I guess that was kind of my process. I think a lot of people have that process. They get introduced to cryptocurrency and Bitcoin and then they start off very skeptical.

(04:55):

And I think that’s a healthy way to be introduced into a field. But then obviously in our space, that tends to be a really bad decision, monetarily speaking. So I got involved in crypto through this underground community called DeControl. DeCentral, there’s some interesting characters that show up there. Like, what was his name? Antonopoulos? What was his name? You know the guy. That Milo guy? Yeah, yeah, no, no, no. He was, he’s still around today. Anyways, Vitalik also showed up there. He actually has been to that little hack space in Vancouver.

(05:36):

And they were teaching Bitcoin and we were just talking about it. And at the same time, I was interested in AI. So the union of those two fields, of those two worlds was always on my mind from the very, very beginning of my crypto journey.

Jacob Steeves (05:51):

So how did you come up with BitSensor? When did you start thinking about BitSensor?

Jon Cole (05:57):

Well, I mean, I got involved with a number of people that were working on BitTensor. So I was one of the main people that was involved in the beginning. There’s Alla Shiv, there’s Taco, there was Yuma. There’s the people involved in BitTensor. I can’t say that I had the idea only myself. And it was also floating around in some of the AI circles. How can we build digital currency style computation? How can we incentivize the creation of AI at the internet level? And I think that there was many different approaches that have been tried and are still being tried. I can’t say that we necessarily won out in this field just yet. I mean, there’s the data parallel and there’s also the model parallel approaches. There’s if you can distribute the model maybe across the internet or you could distribute the data. And those are different ways of distributing the computation aspect of AI into a peer-to-peer system. And I think we still don’t know if where we should be positioning ourselves, I think is the space. Obviously, I have my own opinion. I think that what we’re doing with BitTensor by distributing the data and growing the model horizontally is the way to move it. Funnily enough, we’ll be talking about this a lot at Neural IPS next week. There’ll be a large number of researchers debating such topics about which way should we scale AI into the internet domain. So perhaps we’ll have an answer by the end of a week after next.

Jacob Steeves (07:39):

Awesome. So I like to introduce every project with this really broad and general question. So with that being said, what is BitTensor? What is BitTensor?

Jon Cole (07:50):

What is BitTensor? Great. I’d like to think of it as a market for information. Just so happens that information that is processed by a computer to extract the semantics turns out to be called intelligence. And you can then define it as AI, so it becomes a market for AI. But truly, we’re building information markets at core. So BitTensor is an information market. It’s a way of measuring and trading information, understanding of data. That’s BitTensor to me.

Jacob Steeves (08:29):

Interesting. So with that being said, for people that are very new to this space, can you give a brief, very, very vanilla explanation of what is AI, and how does that incorporate into BitTensor?

Jon Cole (08:45):

I think that the best way to understand artificial intelligence is to use the biological analogy. We even define AI, artificial intelligence, the artificial directing us towards the non-artificial definition, right? So intelligence is ill-defined, actually. We like to claim that we have it, but in many ways, we don’t even really know what it is to begin with. So it sort of exists by its comparison to what we do. So I think that’s the best way to understand artificial intelligence, is to understand what we do. So what do we do is we take in data, usually through our cortices, so our optical nerves, through our visual optic nerve, auditory optic nerve, and then we process that data and turn it into actions.

(09:39):

The intelligence is the whole process there. So it’s the extraction of the semantics of the data, so processing it, fine-tuning, pulling out the signal from the noise, and then it’s also the process of turning that into action and moving to the world. So there’s, I personally believe that intelligence can be well-defined as the extraction of meaning from noise, or the signal from noise, understanding the map from the territory.

Jacob Steeves (10:16):

That’s a really good explanation of it. It’s very, it’s kind of hard to understand for somebody that doesn’t have a grasp on what this is, so I do appreciate that. So with that being said, so what is the basis of the BitSensor network, and can you introduce a little bit about Tau and what is Tau?

Jon Cole (10:34):

The basis is really the consensus algorithm that runs on the chain, which is a mechanism for the computers that make up the nodes on BitSensor to reach consensus, to agree upon who in the network is producing value and who is not, importantly. And then also to what degree, who’s producing more and who’s producing less and producing a scaling or a ranking of those nodes. That consensus mechanism is the core, the kernel of the network itself.

(11:07):

And then that creates, by building that consensus, we can then build on top of that foundational layer of consensus, we can say, okay, let’s define a market, let’s actually release a token based on that consensus in the same way that in Bitcoin, there’s consensus in the nodes around who produced the last block, and then that person is allowed to mint a token. We can do something similar in BitSensor where there’s consensus around who’s producing value and then we can distribute tokens in a similar minting process to Bitcoin, to those computers, thus creating a competitive and incentivized mechanism for the creation of machine intelligence. So if they’re evaluating the production of information, the production of understanding, then that distribution of value to the people that are doing that better creates an incentivization mechanism that creates a market, a way of incentivizing and turning the swarm into a organized cluster of self-interested individuals working together collaboratively to produce something. So that kernel, that’s the basis to answer your question.

Jacob Steeves (12:24):

So can you talk a little bit more about the consensus algorithm and how that creates the incentivization and how people are rewarded in how and how that generates value for the holders and users of the network?

Jon Cole (12:37):

Right, yeah, it’s a very good question. So within the subfield of AI, which is a field of computer science, there is a subfield within side of AI that is the understanding of language, which in order to understand language, you need to be able to understand the semantics of things that are being said. So that’s producing a representational understanding from a sentence, taking a sentence and projecting it into a numerical representation where the meaning has been extracted and the noise has been cut away. The validators with inside bit tensor are effectively determining whether or not the computers in the system can do that.

(13:23):

Now, we write these validators, they’re fairly complex, but the underlying mechanism is exactly that. So they go, hey, can this computer extract the meaning from this piece of text? And if they can do that properly, they effectively, well, they explicitly rank that peer higher. And that explicit weight goes onto the chain. And then that chain then numerically calculates the sum of the weights.

(13:54):

It’s a little bit more complicated than the sum of the weights. We use a consensus mechanism to determine if there’s peers that are cheating and are outside of consensus and whether or not those weights can be trusted and things like this. But the core mechanism is that the validators rank the peers based on how well they perform at understanding or extracting the meaning of the text. That’s our current bit tensor one network. Those weights go in the chain, then the chain distributes tokens.

Jacob Steeves (14:25):

Okay, so with that being said, one of the more attractive parts about the bit sensor network is the lucrative profitability about being a validator, or there’s no minors in this case, right?

Jon Cole (14:39):

Sorry, what do you mean?

Jacob Steeves (14:40):

There’s no like minors in bit sensor, it’s just validators, correct?

Jon Cole (14:45):

No, no, no. Well, the computers that are being evaluated by the validators are the minors. The ones that are producing this understanding of a sentence, for instance, are the minors. And so there are two classes really, there are validators and there are minors. We call them servers sometimes synonymously because in a way they serve a machine learning model, they serve an aspect of machine learning model.

Jacob Steeves (15:15):

And how does one get involved in that? How does one become a validator? How does one become a minor and get involved with the network?

Jon Cole (15:23):

Well, you can enter the network at any time.

(15:26):

And the way that you do that is you solve a proof of work with your machine, which requires a little bit of computational power, and maybe not too little anymore, requires solving a computational problem to prove that you have compute behind your endpoint. At that point, you can be a validator or a server. It’s really up to you. Any peer can validate and any peer can mine. Dr. M and Scrimp are laughing because they understand that that process is actually a lot more complex and difficult than I’m making it sound. There’s a lot of competition to get into one of those slots. And so that actually turns, that turns out to be one of the things that we talk about on the Discord, more than maybe we should like.

Jacob Steeves (16:09):

I was gonna, yeah, go on. What I was gonna say is, yeah, so not, so isn’t there like, there’s like, it’s kind of hard to become in this position. You actually have to do stuff. And like you said, with the computer power. So what would one need in order to solve this problem and even to get started?

Jon Cole (16:29):

Time and a GPU, effectively. You need a computer to mine BitTensor. And in order to solve the proof of work, which is like this initial step into the network, you need a GPU, which can do the hashing algorithm fast enough to out-compete the other people that are trying to enter the system at the same rate as you. Or the same time as you, pardon me. So once you get into the system, then you begin to be evaluated. The computers, the other validators will pick you up. They go, okay, this guy’s here. Great, let’s see how well he can perform.

(17:09):

Then over time, if you perform well, the tasks that this validator has asked you to perform, understanding text, you will stay in the network, hopefully. Now that really depends on the competition of the system. There’s 4,096 UIDs right now. It’s a quite tight network. And so you have to beat out someone else. And that person that’s pulled out is the peer that has the lowest incentive in the network. So this is what makes it hypercompetitive in a sense, because in order to be part of this network, you need to beat out somebody else who may be well more versed in BitTensor mining than yourself. I mean, here you go, we have Scrimp here, who learned by herself with very little computer science understanding how to mine BitTensor from the early days. And she’s now an expert, I’m sure.

Jacob Steeves (18:05):

Yeah, that’s what’s super interesting about the network is that it’s really, anybody can really get in, and it’s the more knowledgeable people. I have a friend that, he’s not in this space now, my friend Justin, he’s been mining BitTensor, and he’s the one who showed me it. And he was saying that it’s like super profitable. And that’s what really got me interested, because I think that the nature of the tokenomics of BitTensor allow the price kind of to go up. And I think that it’s really interesting on the tokenomics side. So that’s my next question, is can you speak a little bit about the tokenomics of TAO?

Jon Cole (18:36):

Yeah, sure. We imbue the token with value by making it the key to the lock, which is the network. So if you want to query the work that’s done by these miners, you need to hold TAO. So that’s what imbues it with value. I can’t speak too much on the price of BitTensor, whether or not it’s low or high. Absolutely. But it exists there as a value-holding token, because it’s what gives you access to the system.

(19:09):

The tokenomics, we attempted to be fairly conservative with it. We just picked 21 million, same emission cycle as Bitcoin. I think that, to a certain degree, that’s fairly arbitrary. I’m a believer in markets, like whatever we would have picked, there would have been a market equilibrium that was created. But I think that people understand that emission curve well. And on purpose, there was no minting to the foundation or any of the early developers, because we really wanted to make sure that the token was non-fiat, that every single TAO that exists holds the value of the work that the individual put into it while mining into BitTensor.

(19:56):

So tokenomics is fairly simple. So if anybody understands how Bitcoin works, they’ll understand how BitTensor works. It’s really not much more different than that.

Jacob Steeves (20:07):

That’s, yeah, that’s super interesting. I like that. I like projects that kind of, I feel like what’s unique about BitTensor is it’s kind of like, it’s an OG project, but it’s new in the sense that its fundamentals are kind of what represents the old crypto value. And this is where I think the crypto space is heading. I think that we’re gonna go back to our values as a space. And with that, where do you see BitTensor being in five years?

Jon Cole (20:38):

Well, I hope it’s a project that’s absolutely alive and thriving. I think that the way that I understand, these communities, cryptocurrency spaces that effectively we’re like, we’re these nation states and we’re full of these different types of people that are all incentivized by the same mechanism to promote the project, to make the project successful, to work, to mine into it, all these things. And I hope that we’re a very large thriving community of engineers, we train our people through the mechanism. And this speaks to the idea that anybody can participate.

(21:21):

Anybody can get better at mining BitTensor. We don’t have to have a degree in computer science. It’s like there’s this kind of like gamification. It’s like a video game in a sense. Anybody can play Call of Duty. Anybody can get better at Call of Duty. You don’t have to go to school to become a Call of Duty player. It’s the same thing with BitTensor. And I hope that we drive that aspect of the project to a much larger group of individuals. It’s always really exciting to see the type of people that are coming in. More and more, we’re getting more classically trained people that are interested in developing the project and really teaming up with others and building like really high quality, high scale, high performance computing systems. And that’s really cool to see. So we’re a collective of companies really.

(22:11):

And I think that’s where we’ll be in 15 years where we have some really hardcore mining teams, some really hardcore mining companies at this point, working inside this mechanism, distributing funds to people in a fair way hopefully, and being competitively selected to produce value for the network. That’s on the mining side. Now on the machine learning side, what I really, really want to see and what we work towards continuously, where I spend all my time, where I hire individuals, is working on the core machine learning aspect of BitTensor to make it a project that anybody can use and that is really, really useful to AI researchers.

(22:57):

I love this technology. I get a lot of value out of it. I’m excited to use the computation that’s at my fingertips because I hold Tau. And I want to make that very easy to use for people. I want to build out the technology, the tooling, so that other people can use it as easily as me.

(23:15):

And at that point, build an AI company, a pretty classic AI company on the other side. So this is sort of a two-sided market in BitTensor. There’s the miners, which are quickly becoming organized in order to maximize their inflation of Tau. Then there’s the other side of the system where there’s people that have Tau and they’re trying to maximize the value of that token that they’re holding. That’s where I’m focusing my time. That’s why I’m going to machine learning conferences to talk to AI engineers and showing them what we’re doing continuously. So in five years, I’d like to say that we’re at the level of somewhere like a stability or an open AI or a Cohere, a big company that has just been classically funded to train machine learning models. But now we’re using this much more large and incentivized collaborative system. That’s where I imagine we are. That’s amazing.

Jacob Steeves (24:14):

And okay, so there’s somebody that literally knows very little of Lex Friedman-level podcast about AI. How does BitTensor benefit these AI researchers and how can they use that? Because I feel like we haven’t covered that enough and that’s an extreme use case. And a lot of cryptos that we see don’t really have use cases like that. So I’d really like to hear a little bit more about that.

Jon Cole (24:38):

Yeah, that’s a really good question. It really comes down to the core technique for that BitTensor is built on top. So we’re built on top of what’s called a mixtures of experts. And a mixture of experts is a type of machine learning model. So it’s a neural architecture where there are a large number of different components that understand a small aspect of a greater problem and then you can unify those smaller components. The value proposition to machine learning engineers is the ability to broadcast into this large peer-to-peer network and extract an incredibly diverse and also pre-computed because you’re not doing the computation. The computation is on the minor side. So to broadcast into this network and get a whole bunch of computation done for you on your inputs and then learn who you want to talk to in order to understand your problem.

(25:32):

So for example, a project that you might know is Cohere or OpenAI and they build GPT-4 or GPT-3 and Cohere has their own language models or StabilityAI has their endpoints that you can query. BitTensor is a collection of those endpoints. It’s a swarm of endpoints. It’s Cohere times 40,000, 4,000 networks. It’s a whole peer-to-peer network that’s incentivized of nodes, of endpoints, of miners that are producing value like these companies. We’re like the broad version of these API endpoints that are produced by like standard classic AI companies.

(26:14):

So we provide, you know, speaking from just a purely like corporate vantage point, we’re the same thing to the customers of those companies. We’re this endpoint, but it’s a large number of endpoints that you can query to extract value. And that’s why, you know, a lot of the work that’s gonna happen over the next couple of years is going to be smoothing that process. We’ve worked so hard really on the mining side to make sure that we understand the system, we can train it, we can move it. You know, what are the miners properly being incentivized? The other aspect, like the continual aspect is, okay, can we build this out like we were a classic company, but there was nothing fancy behind our endpoint. You know, we obviously have something fancy behind our endpoint. We have a peer-to-peer network that is incentivized and there’s the whole flip side of the project. So we’ve had to focus on that. But over the next couple of years, I really wanna focus on the more just, you know, classical, you know, pip does install BitTensor query in to get your representations from your text, you know, to have the same fidelity, use of use that these classical AI companies are producing.

Jacob Steeves (27:25):

Yeah, that’s super interesting. Yeah, that was really easy to break down for somebody that literally doesn’t really understand AI. So with that, another question that I have is, is TAO a P2P network? Is it like, you know, is it P2P?

Jon Cole (27:42):

Yeah. Peer-to-peer usually means that it’s each of the endpoints is run by an autonomous individual organization. Yeah, that’s what I’m referring to. Yeah, so we don’t control the endpoints. They’re run by individuals. Rather than say like a non-peer-to-peer system would be like Google, which is run by a single institution or a single organization. BitTensor is the product of a whole group of individuals that run nodes, maybe more than one. And then the collection is the network itself. So that’s, yeah, it’s peer-to-peer. It’s core peer-to-peer technology.

Jacob Steeves (28:24):

Yeah, that’s really interesting. Cause I think it adds a level of decentralization where it’s kind of hard to have decentralization, but I feel like that at least adds a level to it.

Jon Cole (28:34):

Yeah, there have been some criticisms of us that we integrated too strongly with a blockchain because we’re a peer-to-peer network with a blockchain in the center, right? And that’s like another layer. So we’re not just a peer-to-peer network. A really great project is HiveMind, something that I’ve actually worked with, a great team from Russia out of Yandex, they built HiveMind, which is kind of like BitTensor without a blockchain.

(29:06):

And they’re very, very, very peer-to-peer. So, but there’s no intensifization layer at all. It’s like totally like steady at home, folding at home ethos for AI research. And I always like to promote them because I think they did a really great job also in a kind of different thread or different like a different hierarchy in the space.

Jacob Steeves (29:30):

Yeah, that’s super interesting. I haven’t heard of that project actually, but I’m definitely gonna check it out. The next question I wanted to go to is, can you speak a little bit about the BitSensor team and who’s on the actual team?

Jon Cole (29:43):

Yeah, sure. So there’s a number of individuals that we’ve pulled from the community. One of the greatest things I think about incentivized protocols like this, and the fact that we didn’t do a pre-mine is that we just hire from our community because they all have equity already. And they’re incentivized to work with you already. So we have like four or five individuals that have pulled from the community, some really interesting, really, really intelligent people that are machine learning engineers that just fit our world perfectly and we pulled on. Then there’s a couple engineers we pulled right out of university that had like no experience with AI and then we’ve turned them into like our core team. I won’t name names. There’s no point in doxing them. Of course.

Jacob Steeves (30:32):

And we have a team of about 15 engineers

Jon Cole (30:34):

and that’s growing slowly and methodically. We’re not in a rush to expand too fast. We always need more engineers though. If anybody knows any really great engineers, data scientists that are curious to work with us, we would love to speak with you. We work remotely and then partly also from Toronto. So like the hub is in Toronto, but I’m in South America. We have people in South Africa and Europe in the United States and Canada and yeah, that’s about it.

Jacob Steeves (31:12):

Awesome. So can you speak a little bit about the roadmap for the upcoming year?

Jon Cole (31:17):

Well, it’s gonna start off with a bang with the Finney network, which is really gonna be fun. That’s gonna open up many avenues for us in terms of development into the more classic, decentralized ecosystem. Building things like DEXs and smart contracts on BitTensor to integrate with other types of tokens is gonna be possible because we pushing to poke it up.

(31:42):

That’s gonna be a whole push. Now, from a development perspective, we kinda wanna see the community take on some aspects there because at this point, we really do become connected into an ecosystem we don’t control and people can use TAO on different chains and things like that. It’s gonna be really interesting to see what the community does there. We wanna stimulate that. That’s gonna be the big initial push. Now, the movement to understand and train state-of-the-art machine learning models on top of BitTensor is gonna be a continuous thread and we’re never gonna stop doing that. I think there’s just so much, there’s so much research that we can do on top of this system that we want to carry out. And that’ll be the majority of the work that we do as a company.

(32:35):

Now, there’s the data science aspect, there’s understanding this chain and how it progresses through time. There’s the expansion and the fine-tuning of our consensus mechanism. There’s the adding of new networks on top of BitTensor sub-networks using images, not just text, potentially time series. That’s all on our roadmap. There’s a potential centralized exchange, a listing that we’re not pushing that could happen from a cloud-based perspective. From a community, we’re not so sure. There’s expanding just the chain itself to make it more performant and more UIDs and then making the miners better and all these things are ongoing. I really think that we only just scratched the surface of what kind of innovation we can build in this new domain for AI. And I’m really excited personally, I’m really excited about what the general AI community will say when they can see a lot of what we’re doing.

(33:33):

We can begin to push out benchmarks and results that will interest them in a more academic setting.

Jacob Steeves (33:42):

Awesome, that’s really exciting. I really like to see Tal on different chains and I think that one of the biggest questions from the community that I’ve seen, it’s just like, when is it gonna be on an exchange or a sex? And I think that was a pretty good answer to that question. I think that was a little bit alpha right there, guys, so.

Jon Cole (34:01):

Yeah, I might speak to this point. The foundation itself doesn’t promote centralized exchanges because we can’t and I think it would be irresponsible of us to do that. It’s right for us to stay straight and narrow on the core, you mentioned like the core philosophical foundations of the space, right? Decentralization exists for a reason. And if we forget about the importance of decentralization as a space, we’re straying very far away from why we’re here.

(34:43):

So we like to focus on those core philosophical directions as a project and as a team. We want this project to be decentralized. We want to make sure that things like the pseudo key are not just held by a set of individuals, which just happens to be the foundation. And we wanna make sure that the project cannot be taken down, right? It’s censorship resistant. Currently is censorship, sorry, it’s censorable. Definitely is censorable in its current state and that’s not good. And if we don’t move on those aspects of the project, we’re doomed to fail. And the last little while in crypto with the FTX explosion and things like this, really I think should bring us back to the roots of what is important here. The fact that there are enemies to this space, there’s enemies to our projects and we need to focus on defensive technologies and not get carried away with the expansion of prices and be so gallivant and excited and price focused and things like this. When really what we’re building is a completely new ecosystem of finance and that’s what really we should be focusing on long-term and that will be what wins the day.

Jacob Steeves (36:01):

And speaking to that, I mean, guys, shout out to everyone who’s still here after the FTX situation. I don’t know if that was the true capitulation event, but my God has crypto Twitter lost probably a lot of people and it’s given us a lot of bad name in the space. So that’s what’s really interesting about the crypto space right now is that so many people left, but the only people that are here are the ones that are truly passionate and the ones that are continuing to build and that are in it for the tech. And now we’re here before the prices, right? Yeah. Like me personal, like- Yeah, crypto nirvana, right?

(36:35):

Exactly. And like, and speaking to that, I kind of want to hear what your thoughts on the FTX situation are coming from somebody that’s high up in a project. What are your thoughts on just the situation and the impact that it has on the space?

Jon Cole (36:51):

You know, I’m an optimist. I saw the 2014 crash. I saw the 2017 crash. I’ve now seen this crash and every single time, you know, what happens is that you trim the fat. There’s people that come in here and they’re just here because they want to make a lot of money and they sell scams and they soak up some money and it really sucks and I’m really sorry for people’s losses, but that’s the learning process of life. Like that’s, you need to have the bad days in order for the thing to truly, you know, grow, right? It has to be, we’re anti-fragile, right?

(37:30):

And Taleb’s not a big crypto guy, but his ideas actually do define us in many ways. And these, you know, tumbles, these bear markets are what, you know, make the herd stronger. It’s what brings us back to the core, the core foundational ideas and philosophy of the space. It’s what gets out the loud people and lets the more quiet voices, you know, take the floor. You know, I remember back in the 2017, there was like up in the building, they build up in 2017 in this crypto space I was hanging out. They would all, you know, all of a sudden it’s all the crypto bros, you know, like here they are, they want to sell you your new coin. And I just remember after the crash being so happy that they were gone, you know, now we can actually talk about like legitimate stuff.

(38:20):

So, you know, that’s where we are right now. And it’s great to be surrounded by people that, you know, they’re really here for the right reasons. And this is where we build things. This is where we actually make real progress. So I don’t see it as too much of a bad thing. And obviously FTX was an example of, he was antithetical in every way to what cryptocurrency is about. He was selling a fiat currency, he was a bank. It was, there was a middleman involved. It wasn’t verifiable. It was trustful, like every, none of those things define what we’re doing and what we’re building in this space. So it blew up and, you know, our enemies are going to laugh at us again, but we’re going to laugh back in a couple of years when they discovered that they just, you know, they shot their own guy, right? It’s just proof of why the system that we’re fighting, you know, needs to fall down in the first place. So, yeah, I’m optimistic, but obviously a little bit sad for the people that lost money, you know, my condolences.

Jacob Steeves (39:25):

You know, that’s the optimist perspective. I think that’s, I think you’re, I would say I agree like half with what you’re saying. I’m more of a pessimistic, I’m on the opposite side. I think that this event has delayed the crypto space being legit for a few more years. And I think it gives more people time to build.

(39:44):

There’s just positives and negatives for everybody on either side. So many people have different roles in the cryptocurrency space. So it’s really by situation to whether this benefits you or if it wasn’t, but again, and also I really am sympathy for all the victims who have lost money. I’ve lost so much money in crypto before, believe me. I luckily wasn’t on FTX, but believe me guys, I’ve been there too. So I wanna call up some members to the community now to answer any, to ask any questions and yeah. Hey, Dr. M, how are you?

Dr.M (40:15):

Hi, John. Hi, Konst, I’m doing well, thanks. I was hoping, you know, maybe if Konst could touch on, you know, what it would mean to move beyond substrate and be a parachain, the implications of it.

Jon Cole (40:31):

Yeah. Yes, good question, Dr. M, nice to have you up here. Well, the major change is that we start to have these things called collators, which are merging our chain with the relay chain on Polkadot and this allows us to have a shared consensus where nodes can move things like funds across chains and into DEXs and obviously that’s the, you know, the implication from a utility perspective for us. The other aspect here is that we begin to form this collator set where there are a number of potentially different institutions and organizations and groups of miners that run those collators. So we stopped running the validators on the chain. Right now it’s a proof of authority, so we have basically 100% control over the chain. When we move to these collator sets, we just decentralize that control and there’s different entities that can run that and they’ll be incentivized to do so.

(41:36):

We lose control and we give it to this swarm of people that are running collators. So that creates a form of decentralization for the project and that has implications for us, obviously, because if the chain goes down, right, we have to organize amongst a bunch of people. So it has to be done in a very careful way to make sure that we let loose the chain in a state that we’re happy and think that it can last for a long period of time.

Dr.M (42:11):

Awesome, awesome, I appreciate that. So the interoperability between chains, I mean, that is, will enable so much, will enable that Tau and BitTensor ecosystem, which is an economy unto itself, this is a comment you made a few days ago in a space and I’ve thought about how it’s so true that BitTensor unto itself is a huge economy and of course will hopefully be so much bigger later on, but that’s very exciting. I think a lot of people in just the crypto space in general, they would, well, first of all, they would say get exposed to BitTensor, whereas they wouldn’t have been otherwise when it’s on substrate and then of course will enable a lot of people who would build things having to, around Tau, I suppose that would not do so before parachain. So I’m very excited for everything coming up.

Jon Cole (43:16):

Yeah, thank you, Dr. Jamil, exactly, yeah. That’s the thing that I think excites me the most, that gets me up in the morning is seeing the construction of this economy. I like your word choice there with economy. It really is that way, like there’s BitTensor, there’s the kernel mechanism and then there’s people that are latched onto that, which are then themselves working together to run miners and paying individuals to mine for them.

(43:48):

I know a number of people that don’t even mine BitTensor, they just pay people to mine BitTensor. I know a number of people that have created companies and formed them in different tax havens around the world to mine BitTensor. I think that’s fucking amazing. That really gets me excited. I know there’s an individual that’s mining Tau for an incubator program out of their university. So they’re actually making money for a university through their incubator program. I think that’s amazing. I love to see the spread and the growth of these different organizations and just the organization of people around the core market of BitTensor and how that’s coming into something that’s really complex and interesting.

Jacob Steeves (44:40):

Can I just say how cool that is that a university is generating revenue using BitTensor already? That is the future right there. And I do remember hearing about this, but I do forget which school. Not gonna dox them, but I do remember hearing that. And another thing I wanted to add is like, yeah, like guys, BitTensor is its own economy. I have a friend who literally, he quit his job to literally set up Tau miners for people online. Really? Yeah. He quit his job and was setting up miners for people and it was very lucrative.

(45:13):

And that’s where I first really took an interest in Tau. And I was really interested because I thought of it as an economy because I was like, huh, this could be a really interesting thing because it’s just because of that itself. But with that, I’m gonna take the time to self-show BitBusiness real quick, which is my business. So if you guys are looking to start a Tau-related business or BitTensor-related business in the United States and need a legal entity, choose BitBusiness for the fastest formations, most compliant businesses, and we will help you through write-offs if you are a miner. So that could be very beneficial to anybody who is in the US and wants to get some of their write-offs with their miners. So after the self-show, I’m gonna bring up MIT. And thank you, Dr. M.

Dr.M (46:02):

Well, of course, thank you for doing this. You know, I feel like spreading the word on something so amazing that is relatively unknown. You know, I feel like we’re just the entirety of more of the world’s people to know about BitTensor. Of course, you know, day two would find all the amazing things that caused the rest of us in the community to love this project so much and increasingly so as time goes on.

Jon Cole (46:29):

Oh, no, more mining competition.

Jacob Steeves (46:32):

Right.

Jon Cole (46:37):

It’s bittersweet. It’s bittersweet.

Jacob Steeves (46:40):

Hey, MIT, I think we lost you. I’m gonna add you as a speaker.

MIT (46:43):

Oh, hey, what’s up? All right, so my question is, so as I understand it, validators query miners for information and get outputs from the miners. How are validators exactly making use of this information? Like I’m running one application, like I’m running one at the moment and maybe I’m not using it to its full potential, but I’m not seeing like a log of information that would be of any use for me or I don’t know if I’m thinking about it right or wrong.

Jon Cole (47:16):

Yeah, that’s a really good question. So the validators don’t exist to use the information, right? That would be more like the client side. They exist to just rank the miners themselves and see if they’re doing a good job. Now, while they do that, they rank the miners individually. So they go, okay, how performant is this miner as an individual? They also combine the responses from the peers to see how well they do collectively. And that’s, you could say, that’s where they use the information is to collectively join it and then rank it in that synergistic way.

(47:57):

But the validators are not incentivized to use the knowledge for anything in particular. So they just throw it away, right? In fact, they purposely erase their models periodically so that they have a continually updated and fresh perspective on the network. If they were to sort of train to a local minima or like train converge onto a particular perspective, then they would not be dynamic enough to pick up new models that are coming into the system. So they purposefully eliminate the work that they’ve done in learning who’s valuable in the network over time. So that you could say, that would be the product of a validator and they just throw that out so that it can be more dynamism in the system.

(48:39):

It’s the clients that would sit behind the validators. For instance, we have a key called Tiberius, which is the largest key on the network. And it’s used by the foundation for doing machine learning experiments. That’s a validator that actually has a validator key. It has tau, it’s validating the system. But because that key is validating, it gives the clients, the foundation, people that in the company in Toronto, the ability to query the network at high fidelity to make a large number of requests and get those responses out really quickly. To the chargrin, to the disappointment and sometimes anger of a number of people in the community who find that those queries disrupt their mining.

(49:20):

But the only reason they’re able to do that is because that key is validating. And so to answer your question very specifically, the validators don’t produce anything. They just check to see that they could be producing something from it, if that makes sense.

MIT (49:35):

Yeah, yeah, thanks for the answer. And another question is, so I’m also trying to wrap my head around. I’ve heard this analogy of Bitcoin being a super computer because you sort of harness the computing power from like nodes across the world. Yeah. Oh, fuck, my alarm just went off. Oh my God. No, bad timing. Okay, sorry. I’m half deaf right now, but, so my question was, so yeah, so that Bitcoin sort of harnesses this computing power to, yeah, to sort of create this, yeah, nevermind. But BitTensor uses decentralization and the power of having like many nodes to sort of process more complex models. But do I have that right, that thinking right? Because I’m sort of struggling to see, right now miners are already the best, the best GPUs like A6000s, A100s that are sort of accessible to most people or the models you’re able to run on them are becoming too big and we’re at such an early stage still. Yeah.

(50:53):

And like, how is that, and does multi GPU solve that whole issue? Because I’m not seeing how, because right now it’s all done on like a per miner basis and I don’t see how we’re using the collective computing power to run complex models. I don’t know if that makes sense.

Jon Cole (51:12):

Yeah, so, well, there was a couple of things in that question. Yes, multi GPU will probably exist at a certain point. You know, I know for instance, that there are people running like a DGXs now on BitTensor, which are like huge compute clusters with like multiple, you know, GPUs, like 10 to 15 A600s, right? Big, big IPU cluster units, things like this, like graph cores and stuff are coming to BitTensor. Now, they’re individual, the nodes are individual. So they’re not connected laterally.

(51:50):

Right, sorry, not laterally. They’re not connected horizontally. They’re connected through the client. So it’s the client that combines the responses from the miners and then uses that combination to create a model that is the collection of the miners on the network and itself. So it’s like, think of it like a bit like a pyramid, right? You know, there’s a fan out. So you send the message to a whole bunch of nodes in the network and you get the responses and you join them. That’s the mixtures of extras model I was talking about at the beginning of the call. The idea of distributing or dividing, conquering the problem and then joining the results.

(52:27):

So BitTensor is one model from that perspective. The individual models are just components, aspects of that larger model that we join at the validators and at the clients to see how performant the entire system is together. So from the perspective of an individual miner, it seems like they’re just running a single node and that node is not part of anything larger. But from the perspective of a client or a validator, those individual miners, those individual endpoints come together to construct a larger mixture model, it’s called.

(53:08):

So yes, we are a neural network that is composed of all the nodes, all 4096 nodes in the network that is probably a trillion parameters. We don’t know for sure. We know for sure that it’s more than 300 billion, but we don’t know if it’s larger than a trillion yet. That’s kind of the black box aspect of BitTensor we can’t really determine, but some people have Prometheus on so we can know how big it is and it’s greater than 300 billion.

MIT (53:46):

All right, thanks. And then did you answer, how are you gonna get around this sort of cap, the computing power cap as BitTensor grows and-

Jon Cole (53:59):

Oh, so like on the individual endpoints themselves?

MIT (54:02):

Yeah, just from personal experience, like yeah, I have like a A6000, A100 and I can maybe fit a couple of models, but as this token sequence length grows and other parameters grow and the demand grows- Multi-GPU is the answer. Yeah, that’ll sort of fix everything. And would you be able to coordinate across like, so not only personally having multi-GPUs, but coordinate across other nodes?

Jon Cole (54:31):

Yeah, that’s a really great expansion to the protocol. You can think of it almost like pooling, right? Yeah. And that’s something that we’ve looked at a number of times is the people have asked for it. It’s just sort of like a development challenge that people need to do themselves. Now, we’ve thought of various ways of doing it. It nests itself underneath BitTensor. So it’s not core BitTensor. It’s like BitTensor would exist at a higher level and somebody could come up with a way of pooling and that wouldn’t actually be integrated in the core technology. It would just be something that’s built underneath it.

(55:09):

It really comes down to someone designing that system. Now, if you learn Torch, the language Torch, which is what BitTensor is built on top of, it’s a Python library for doing distributed machine learning work. In Torch, like at the core of Torch is an API for building distributed machine learning models. So you can have models that are run or inferenced across multiple GPUs. It’s just a matter of time before somebody builds that. And in a way, I’m kind of hoping that the community does it because it really just would be as simple as getting somebody that knows how to write code in Python and Torch to just take a look at what we do with our server and then distributing it across multiple GPU. We do this stuff. We actually, we write additions for the miners, but the way we think about it, it’s kind of like we should probably be putting most of our effort at the client side because the innovation on the miner side is incentivized already by the foundation, right? Or by the protocol, pardon me.

(56:18):

So there’s already a reason for people to go out there and design miners that run across multiple GPUs. And there’s not so much incentive for people to build like client facing technologies because that’s not being incentivized by the inflation of Tau yet, or likely ever.

(56:39):

So anyways, that was a long-winded answer to your question. But yes, the answer is that you’ll be able to run larger models across multiple GPUs and that’s all very possible with inside the tensor. And that’s the answer. I mean, you can scale as much as you want. So you can run 100 billion parameter model across 50 GPUs or whatever, and that’s how people will scale at that level.

MIT (57:06):

Awesome, thanks for the detailed response.

Jon Cole (57:09):

Yeah, no worries. Thanks MIT, it’s a great question.

Jacob Steeves (57:15):

Thank you. So with that being said, I think we’re gonna wrap it up here. So Konst, do you have any final thoughts or remarks you’d like to share?

Jon Cole (57:26):

No, I think that’s all I have to say tonight. I really appreciate you bringing me on. John, it was super fun. Really cool to connect with crypto OGs, and anybody in the space that’s interested in BitTensor is a friend of mine. So thank you so much.

Jacob Steeves (57:45):

Thanks, guys. Thank you so much for coming on. If you’re listening to this, thanks for listening. And if you’re, give me a follow, shoot Konst a follow, and let’s continue to watch what happens with BitTensor over the next few years, guys. So it’s exciting to be a part of this and looking forward to connecting with all of you, whether they’re in the BitTensor Discord or the Passive Income Research Group or.

Video Description

SUBSCRIBE THE BITTENSOR HUB FOR EVERYTHING BITTENSOR!

This Clip was recorded in Jon Cole’s Twitter Space Chat on Nov 25, 2022

Host: Jon Cole Co-Host: Jacob Steeves (Const)

Subscribe to the The Bittensor Hub for everything Bittensor!

Bittensor is an open-source protocol that powers a scalable, globally-distributed, decentralized neural network. The system is designed to incentivize the production of artificial intelligence by training models within a distributed infrastructure and rewarding insight gained through data with a custom digital currency.

Discord: https://discord.gg/Qv3fxVaXyE Website: https://docs.bittensor.com/ Whitepaper: https://drive.google.com/file/d/1VnsobL6lIAAqcA1_Tbm8AYIQscfJV4KU/view Network Map: https://bittensor-explorer-staging.netlify.app/

Socials: Bittensor: https://twitter.com/bittensor_ Jon Cole: https://twitter.com/Joncole23 Jacob Steeves (Const): https://twitter.com/unconst1

DONATIONS: 5DHYPY9P8VBSpp6Q8m8hHrq4ME5UTiXfXVFbH32MNV4JcSxH

HASHTAGS: #BITTENSORTAO #BITTENSORMINING #BITTENSORCRYPTO #BITTENSORNETWORK #AI #artificialintelligence

Sponsors

Podscript is a personal project to make podcast transcripts available to everyone for free. Please support this project by following us on Twitter.