Watch

Transcript

Disclaimer: The transcript that follows has been generated using artificial intelligence. We strive to be as accurate as possible, but minor errors and slightly off timestamps may be present.

Dr.M (00:01):

All right. Hello, everybody. Welcome to AI Sessions with BitTensor. My goodness, it’s been a while since I’ve hosted. I feel like it’s rare that I even speak in the spaces these days. Thanks to BitTensor, I have actually completely been offline anywhere other than the BitTensor Discord for a while. Hi, Ala. Thank you so much for joining us, for giving us your time. How are you?

Ala Shaabana (00:25):

Of course. Thank you for having me. I’m doing well. Glad to hear that we kind of took you away from everything else. Oh yeah, and not surprisingly.

Dr.M (00:30):

Gosh, just reading about BitTensor for the first time ever. Actually, very initially, Ala, I think I’ve told this story a few times. I was wondering, is this like a made-up story in the way of a scam? Because it just seems too good that such a thing exists. But of course, it exists and has been operating for a year. Ala, for our audience, I just should say that Ala Shaabana is one of the two original founders of BitTensor, which is a decentralized internet-scale neural net that is incentivized via the blockchain. Last time, we kind of started by talking about, or me asking you and Jacob, the other co-founder of BitTensor, what a neural net is. I would love to get into some AI general topics, but I think that just what BitTensor is kind of deserves some a bit of attention. But before that, I should say that this will be a regular, bi-weekly space.

(01:48):

I’ll touch on later on, everything about BitTensor is entirely transparent. Of course, the blockchain is transparent, but also the code is open source. And beyond that, the team holds on their Discord a bi-weekly question-and-answer session where I suppose actually anybody from it could join the Discord and attend that and ask any questions. Now that I’m a few weeks into participating as a miner of BitTensor, I’m really impressed by how open the team is and how transparent it is in every way that you can imagine. So, Ala, to start, neural nets have been around since 2012. For those of the audience who may not know what a neural net is, I would refer you maybe to the prior session we had with BitTensor, and that’s in my timeline. That being said, neural nets have been around since 2012, and really I should say they are the reason behind the intelligence of any AI that we see out there. Even something as simple as facial recognition on your phone to unlock your phone, all the way up to more complex things, all of this is really possible because of neural nets. So it is, I suppose, the standard architecture from which we derive the intelligence that we see from AI as of late, past 2012. So, Ala, what is the big deal about BitTensor as a neural net?

Ala Shaabana (03:30):

Yeah, absolutely. I just want to make a correction really quick. Neural nets, I think a better way to say it, they were popularized back in 2011-2012. The first artificial neural network was actually invented in 1958 by, I think it was a psychologist named Frank Rosenblatt, I think it was Frank Rosenblatt, and he called it a perceptron. Effectively, back then, they didn’t have the compute that we did. They didn’t have, they had what was the equivalent of an abacus today. So it was mostly a mathematical model, that’s all it was. And then in 2011-2012, basically, some really smart folks out of, I believe it was University of Toronto, and eventually they kind of joined Google as well, and basically the AlexNet, which was a neural network that was properly implemented on a computer and was actually able to kind of do all of the cool things that, you know, it kind of started the revolution of us doing all the cool things that we could do today. So at the time, it was really good at telling different images apart. It was called the AlexNet.

(04:35):

Now, ever since those days, ever since 2012 and onwards, there’s been an explosion in research, right? There’s been an explosion in funding. There’s been an explosion in people learning how to create neural networks. There’s entire Coursera specialization courses on how to build neural networks. And it’s gotten to the point now today, back in 2012, if you wanted to build a neural network with code, it took a lot of code. It took a lot of work because there’s no implementations. Nowadays, you can actually deploy a small one with a few lines of code and you’re done, because everything’s abstracted away and it’s very easy. Now, one of the things that happened, though, is that, you know, obviously they’ve been commercialized, obviously they’ve been used in R&D, in all these massive companies. These neural nets ended up becoming massive, right? Because they’re becoming more intelligent and more interesting, and they’re doing much more sophisticated tasks to accomplish very specific goals for very specific companies. So, for example, Google or Facebook or, sorry, I guess Meta, or OpenAI or any of these massive companies nowadays, they have such complicated tasks they need to do that they have to use these massive neural networks. But what this kind of did is, in addition to innovation, it also starved innovation from the little guy, right? So, for example, if I’m a, at least in my days when I was a PhD student, I could not compete with anything that Google has deployed. There’s no way, right? I don’t have the resources, I don’t have the funding. Even as a professor, I won’t have the funding, unless I’m collaborating with them in some way. So, it’s kind of created this gap, right? The other pit that’s kind of somewhat of an annoyance, well, not an annoyance, actually, more of a real big problem in AI, and it’s that knowledge does not compound, right? So, all this research being done, and this applies everywhere, but it’s especially a little bit embarrassing in AI, because we actually should be better than that. It’s that we’re building on top of previous work, right? So, if I published a paper today, and it was some really interesting neural network, you know, Jacob, for example, my co-founder, could publish something even more interesting next year, and he’s built on the work that I’ve done. So, it’s kind of compounded the knowledge that I’ve created, and kind of created something that’s even more interesting next year. So, it’s kind of compounded the knowledge that I’ve created, and kind of created something interesting. However, the network that he built to beat mine, right? Let’s say he’s utilizing the work that I did. He has to still train from scratch, even though my network has already learned a lot of the knowledge that his network needed to know. There’s no sharing of knowledge. There’s no way to build on top of what’s already existing in terms of computational knowledge itself.

(06:52):

So, one of the cool things about BitTensor is, and that’s kind of where it comes in, is it tries to kind of resolve these problems. It actually creates a compounding of knowledge within the network. So, you no longer really have to train everything from scratch, the knowledge is already there. So, what happens is, when you deploy any neural network that you have on the BitTensor network, it will learn from the other networks who’s most useful and who isn’t, and start communicating with them, and kind of take advantage of what they’ve already learned to make itself smarter. Does that sort of answer your question?

Dr.M (07:20):

Yes, absolutely. So, I’m sure that it sounded like the way that you met Jacob was that you somehow met each other and realized that you’ve been thinking about the same problem for a while. And then, of course, since whenever that happened, there has been quite a bit of work. The reason I’m getting into this is because I was about to say that BitTensor began operating from ground zero about just one week short of a year ago. But, of course, quite a bit of work happened before that could happen. I’m wondering, Aloy, if you can give us a sense of the progress that you have seen in one year of this network being alive. And basically, if you try to give us an understanding of how it compares to the rate of development or development of intelligence in already established paradigms that you touched on, like academia or industrial development by tech giants.

Ala Shaabana (08:27):

Right, absolutely. So, the one caveat I want to put down before I say anything else about the network and its progress is that this is still fairly new. So, I’m not going to say anything like, you know, we’ve beaten the development of current existing neural networks, because we’re not really quite there yet. But what is interesting about the BitTensor network since it was deployed is what I want to, I think probably our machine learning team might hang me for this one, but what is basically called the synergy between the models. The way that they interact with each other and the way that they learn from each other has actually evolved quite a bit compared to November 3rd last year when it first launched. This is, of course, partially to all the code updates we’ve made, partially due to all the hyperparameters changes that we’ve made and everything like that. But the interesting thing about it is that they’ve gotten much smarter at being able to tell and distinguish who’s useful to them and who isn’t. So, one of our resident senior engineers, Joey, he’s actually been working, well, he’s done this a little while ago now, but it’s kind of an interesting result that we found. It’s very, very promising. He deployed a model that speaks only German. It does not speak English, and it only learns in German, and it only understands German. And what that model did is it actually was able to discern from all the like 4,096 different models on the network, was able to find like two or three that were actually able to speak and understand German with it, and was actually able to speak with them and understand from them. And what that indicates is that these models, they don’t know what’s out there, but they’re able to discern on the BitTensor network since a year, basically for the past year, we’ve actually been working towards this goal of being able to discern who is useful and who isn’t. And that’s actually one of the main hypotheses of BitTensor, is allowing models to speak for themselves. Don’t let people speak for the models themselves, because that creates a lot of bias in the way that we pick the models, and it creates a lot of problems with the way that we develop them eventually, right?

(10:20):

Right, right. You know, one classic example that I like to use, and that’s because I’m coming from academia, is peer reviews, right? The amount of times that peer reviews have been marked as fraudulent, or they’ve been marked as problematic, or the results were funny, one of the most famous problems in AI as well is reproducibility. I cannot reproduce a paper that I see. And that’s why websites like Papers with Code have popped up, that will really rate papers that actually deploy their code with the paper as well, so you can actually get the results. Now, the other problem with this is that if I went to publish something, let’s say I publish some interesting model, and I want to publish it in one of the highest, most peer-reviewed conferences in the world, that’d be great. And then if someone else who’s significantly more famous than I am, let’s say he’s a really interesting, a really powerful AI engineer, AI scientist, and is very well-known, he publishes exactly the same thing I do. More likely than not, even though the reviews are like double-blind, his paper will be picked over mine, because he’s much more influential, he’s much more well-known, and that’s kind of a given, and that’s just how humans work. Now, the interesting thing about BitTensor is that who I am is no longer relevant, right? That’s kind of part of the beauty of Web3 as well.

(11:27):

It only matters how your model performs. Is your model doing well? You’ll be rewarded. Is your model doing badly? You’re not going to be rewarded. Is that so?

Dr.M (11:36):

Yes. Yeah, BitTensor is very impartial. I’ve thought about this quite a bit, about how BitTensor is made basically to accelerate development of this intelligence.

(11:47):

Very transparent and very impartial and unbiased. To make a correction in my question, I didn’t mean, I think it would be very, very unrealistic to expect BitTensor at this moment to be the world’s most intelligent neural net or anything like that, but it seems to me that the rate of development of intelligence in BitTensor far outpaces what has been developed out there in the way of neural nets. Am I correct in that understanding?

Ala Shaabana (12:22):

Yeah, absolutely. You can actually, I think even a better way to phrase it would be the size, the collective size of the network has exploded compared to what’s been out there, right? So as a really famous example is GPT-3, which is the kind of the state of the art. Well, it was state of the art. I think GPT-4 is coming out very soon, but it took them $12 million to train GPT-3, and it didn’t even train on the entire data set that they gave it. It trained for I think three quarters of an epoch or something like that. And it was an insane amount. It was so insane that the company that created it, which was OpenAI, had actually gone from a non-profit to a for-profit to sell a portion of itself to fund its training. It’s just crazy. And on BitTensor, if we take the collective size of the entire network, we’ve actually surpassed GPT-2, which is their previous iteration, and we’re actually close to surpassing GPT-3 already in size. And you can actually take advantage of all these models without having to pay millions and millions and millions of dollars, right? Now, again, of course, the caveat goes into being that this is an early project. There’s a lot of work to be done in terms of the quality that you’re going to get out of the project itself, but that’s only a matter of time more than anything.

Dr.M (13:34):

Sure. Sure. That’s amazing. I think the last time we talked, which was maybe about a month ago, we were talking about this and you were saying how it took BitTensor about six months to get to GPT-2 level, which is something that took a lot longer than that. And also, just for a bit of an understanding of how big this is, when they made GPT-3, they actually made it a little bit bigger. When they made GPT-3, it had to be trained again on everything that actually on this, if I’m not mistaken, on the same data set that GPT-2 was trained on. And what Ala means by retaining that training, that’s a huge problem that BitTensor solves is that the network retains any prior learning, whereas currently there’s lots of resources being spent on basically retraining the next iteration of a model. Is that right?

Ala Shaabana (14:30):

Yeah, that’s exactly right. So, futuristically speaking, if, let’s say, BitTensor had been deployed and running, say, five years ago, and then OpenAI had trained their GPT model on BitTensor, their next training, A, it won’t cost as much, and B, it would just be deploying on top of what their previous training has done. It would not be starting over, which they’re doing for GPT-4, right? They’ve kind of started all over again, and they’re making a much larger model, and et cetera, et cetera, to kind of get the model weights that they really want out of the system.

Dr.M (15:02):

Right, right. Ala, Jacob mentioned a while ago about there are people that like incentivized development of intelligence and people who are not agreeable with it. I’m wondering, to me, it was an obvious, tremendous advantage, because when I look at the size of Bitcoin as a computing network, the only reason that it has gotten bigger than, say, any one entity in the world could afford to own, let’s say, or to have that large of a compute network is because of a popular incentive, basically. It’s because of people across the earth having financial motive to join and to contribute to it. So, I’m wondering, to me, it seems like a no-brainer that an incentivized system is what we would need to kind of significantly surpass in time the current capabilities in terms of neural nets that we have out there that have been developed typically by entities, by large entities, or larger or not so large. So, is there a disadvantage with incentivizing intelligence?

Ala Shaabana (16:23):

That’s a very good question. So, incentivizing intelligence itself is not, obviously, it’s not a silver bullet. There’s going to be some problems you might run into eventually down the line with what you’re doing. So, one of the issues is that because it’s open source and open everything, you might have folks trying to take advantage of the system to build something that’s nefarious. And that could be anything from an evil model to do evil things or to a model that tries to take advantage of the system and how it runs. And that kind of opens it up as a bit of an attack vector. Now, obviously, this is a much more difficult problem to actually solve than it is to actually say. So, building an evil problem to take advantage of the network is very, very difficult to do. That’s something that we’re kind of looking into slowly, but we’re not even near that. But if somebody wants to deploy something evil on the network itself and wants to do something bad with it, with that trained network, you can’t really do anything about it. Because that is a bit of, it kind of comes with the territory of being in Web3. However, having said that, that problem still exists today. I can still create an evil model anyways, right? And I don’t train on the TensorFlow, I could train on somewhere else, it’s going to take me longer to do, but it would still exist out there and it would still be an evil network that I’ve created. So, it’s kind of a give and go kind of thing. The other kind of potential disadvantage with decentralized AI is the fact that, and it’s actually something that we had a brief chat with some very, very smart folks about recently, it’s that because the models are black boxes, you don’t have insight into the architecture that they have in the back, right? So, for example, let’s say, Dr. M, my model is speaking to yours and you’re speaking to mine, we don’t have insight into what it looks like on the inside, right? It’s more dependent on your altruism, right? It’s more dependent on your altruistically motivated goals to actually publish what your architecture is. So, what they do nowadays is, you publish a model on Arxiv or on whatever conference that you have, and then you would basically publish architecture for that. But what the companies normally do is after they publish this, they would not publish the weights. So, the weights of the model itself, which is basically, you can think of it as the tuning within the model, that is always proprietary.

(18:30):

That is not for something that somebody releases. So, the folks who made GPT-3, they didn’t publish their weights of how they tune their GPT-3. That they leave up to the rest of the world to kind of deal with. Their excuse, I believe, was that it was too powerful to give to humans and they were afraid of yada, yada, which is a great goal, but we’ve got to take out the grain of salt, right? Because it’s for commercial purposes. So, the same thing with BitTensor applies in the sense that because I don’t have insight into what your market model architecture looks like, I’m a little bit stifled scientifically because I can’t see what it is. However, this doesn’t stop you from actually publishing architecture if you wanted to, to the rest of the world. Because the architecture itself, it’s great and everything, but it’s not really as important as the weights of your model. So, in a sense, it’s kind of a give and take problem. It’s not a silver bullet solution in that sense, if that kind of answers your question.

Dr.M (19:26):

Yes, yes, yes. No, absolutely, yeah. Is this related to Prometheus or is that completely unrelated?

Ala Shaabana (19:37):

No, it’s completely unrelated. Yeah, Prometheus is actually more of a way to give visibility to the rest of the network, but we still don’t know what your model is, right? So, I’ll be able to see, for example, what your output’s looking like. I’ll be able to see all kinds of information like what the model parameter size is, if you pulled it from HuggingFace and so on. But I don’t, for example, if I went and created a custom model myself, I didn’t pull anything from HuggingFace. I started from scratch. I was really sophisticated. I can maybe hide what that model looks like, right? That’s not to say that it’s anything nefarious, right? So, I still have access. The entire network still has access to your outputs. We still know what your model output back to us because that’s part of how BitTensor works. You have to give us back an output, but we just might not necessarily have access to see what the inside of the model looks like. Now, Prometheus itself is actually almost like a… You can think of it in a way as a God’s eye kind of tool. It’s a way for us, us being the users of BitTensor, to see what BitTensor is doing, what the network looks like, right? How many hyperparameters are we sitting on right now? We don’t really know until we use Prometheus. What is the quality of our logits? What is the quality of our embeddings coming in and out of the network? We still don’t have that information.

(20:46):

So, all of that is to kind of put it into a collective little dashboard for the entire network to kind of play with and look at, and it also gives you local information about your own model and your own miner, right? So, you can also deploy Prometheus the same way, but you can also just look at your own local statistics that might not be visible to the rest of the network, because, for example, you want to keep it private, or you want to see how your own performance is doing, because at the end of the day, every user is being competitive, right?

Dr.M (21:09):

Right, right. Ala, something in my mind is pulling me away, and I just need to state it. So, first of all, everybody here and whoever is listening to this, full disclosure, I began mining BitTensor the moment that I really realized what it is, because it seemed very foolish not to, and in fact, I’ve kind of rearranged my life to be involved in BitTensor. So, actually, I just want to say that if you’re in this space and you’re hearing about BitTensor, it is kind of, in terms of a financial incentive, to me at least, and this is not by any means an official BitTensor space, so I’m just expressing my opinion. It is sort of equivalent to having heard of Bitcoin in about 2012. So, I began mining BitTensor because I’ve always been a nerd, and actually, the idea of learning to do something that I didn’t know how to do before, and that it was a technical thing, this was very appealing to me, just by personality. But I would say that even the opportunity, because I know that you guys are not really, you don’t have a marketing department, you haven’t been focusing on creating hype by any means. In fact, if anything, it’s been very low-key, so I just want to say to our audience that if you’re hearing about BitTensor, just the opportunity to just invest, even if it’s a tiny bit, will likely go very, very, very far, and that this project is very early, and really, when I say you guys don’t do any marketing in my mind, I just feel like BitTensor won’t need it, because I feel like for anyone who remotely has any understanding of, just even conceptually, of what AI is, and how AI works, when they read about BitTensor, it should be very obvious that, oh, it should be very mind-blowing that it even exists. I notice a lot of the models that I personally have been using to train on BitTensor have been GPT-3-based, so actually, one of my questions was going to be, is the network at around this stage at GPT-3 capability linguistically, or not? But I think you kind of touched on that, if there’s anything further.

Ala Shaabana (23:35):

Yeah, no, I think we’re fairly close. It’s safe to say that we are getting fairly close to GPT-3’s linguistic ability, just of the sheer number of networks that exist out there. Right? However, to beat it, or to achieve state-of-the-art, that’s a different story. I don’t think we’re quite there yet. This is our Cortex teams, kind of, it’s their territory, they’re the ones who are kind of working on the validation, and benchmarking, and kind of reaching state-of-the-art. And once we do, I promise you, we’ll be telling everybody about it. But at the time being, for now, yes, we’re entirely GPT-3-based, and so collectively in size, we’ve definitively beaten GPT-3’s size, I think they’re at 175 billion. I think for us, conservatively, we’re at 200 billion, probably, but size does not always equal power, right? In our case, because we’re so decentralized, it’s a whole other problem to test and to validate this network working as anticipated for us. And that’s kind of partially also why we’ve capped the amount of miners on the network to be at 4,096, just so we can kind of have some semblance of control, and kind of have more of an environment that we can test things on.

Dr.M (24:44):

Right, right, right. For our relatively general audience here, those numbers are referring to the number of parameters, and the models are sized that way too, so how many parameters they have. And generally speaking, the more parameters, the more intelligent a network would be, or the more intelligent a particular AI model would be.

(25:12):

Ala, what do you see stands in the way of BitTensor? Because to me, it’s by all indications, all that BitTensor needs is just time and the continued hard work of however many you are there in Canada developing it. I’m wondering what stands in its way, what are the major roadblocks?

Ala Shaabana (25:35):

We’re actually scattered all over the world. So we have Canada as kind of the base camp, but we have an office in Austin, we have folks, our head of AI is out in South Africa, our infrastructure engineer is out in Spain, so we’re kind of all over. But yeah, actually, it’s a very good question about what stands in the way of BitTensor expanding all the way out. To be honest with you, I think the one thing that does is powerful people taking advantage of the network. That’s probably the best way to place it. So myself and Clarence in the foundation, all of our goals are long-term. We’re all after the ultimate goal of decentralizing AI properly, creating this network that is beautifully crafted, that works as we expect and as we want it to, and that is creating this amazing AI. But for example, if you have some massive entity that comes in, tries to kind of take advantage of the network to kind of speculate more on the Tau and kind of earn more and yada, yada, yada, it’s a very difficult thing to do nowadays because there’s so many people on the network now that we’re kind of past that point. But somebody acting non-altruistically to the point where they harm the network, that’s one possible problem that could stand in the way. And the only people who might actually end up doing that are the people who are actually, they have a lot at stake. So they’re people who kind of want to take advantage of the whole thing.

(26:50):

Thankfully, we’re kind of decentralized past that point for now, so we should be okay, but we still have safeguards in for this. The other bit that could stand in the way is, to be completely frank with you, is us not being able to produce the results that we are trying to produce. Us not being able to show that decentralized AI works the way that we expect it to, or that us not being able to achieve state-of-the-art because of some intrinsic problem that is within the research. However, this is, again, another thing that is, thankfully, as far as we’re aware, as of this podcast, is minimized because we’ve been validating our research, we’ve been validating our work. It’s actually a little bit premature to say, but we’ve actually gotten accepted to Neural IPS, one of the bigger conferences in AI, so it’s more validation for the work that we’re doing. And we’re getting a lot of really smart people kind of evaluating our system and looking into it and seeing how we’re doing and kind of looking at it with a critical eye. And always the output is, you know, this makes sense, however, fix A, B, C, which is a given when you have anybody validating your work. So, thankfully, knock on wood, we’re kind of doing all right on that side as well. So, really, the problems are more engineering more than anything.

Dr.M (27:58):

Okay, right, right. I just remember having read something about the UID expansion, and for the audience, the UIDs, Ala mentioned the 4096 number, that’s the number of basically available mining spots and mining in the tensor is serving the network with an AI model, and currently it is in natural language models or text generation models, transformers, that sort of thing. The question I was going to ask is, wow, actually, the question actually just escaped me, but that’s okay. Let’s talk about multimodal models. Will that ever be a possibility on BitTensor? Oh, of course, 100 percent. That’s how you end up with interesting stuff like stable diffusion, right?

Ala Shaabana (28:46):

But we’re kind of building the building blocks, right? We’re building the building blocks towards multimodal models at this point. So, as for the general audience, we’re only working on text-based models for now. So, any data that is text-based, whether it’s in any language, that’s kind of what we’re working with. We’re not doing anything like images, videos, audio, and so on, just because we want to make sure that we can solve the problem correctly and concisely for text-based models. So, we’re not doing anything like images, videos, audio, and so on, just because we’re not doing anything like images, videos, audio, and so on. So, we’re not doing anything like images, videos, audio, and so on. Once we’ve done that, then it becomes just really a matter of expanding and optimizing what we do towards other modalities like images and audio. Once we’ve done, for example, let’s say we’ve moved into the next bit that we want to move into would be images. So, if we solve the problem of images as well and we get the images working as we expect and the network that runs the images is running as we expect it to and is producing the right output, then it’s time to create multimodal models, right? Because what you would end up with before we introduce multimodal models is kind of subnetworks, right? You have a subnetwork, or like a branch of the tree, that goes towards text, and a branch of the tree that goes towards image. And what we wanted with multimodal models, because they work with two modalities or more, so you can have a model that works with text or with image, or a model that combines the two, you want to have those in the middle. So they’ll kind of start bridging those networks together, because they work with both text models and with image models. So they kind of act kind of as intermediaries in the middle, and that’s how they get deployed eventually.

(30:10):

So it is, to kind of get back to the question, it is definitely on the horizon. This is something that we’re working with. But we want to make sure that we solve the problem appropriately for text and for the next modality, and then we’ll deploy the multimodal model accordingly to the text and the next modality as well.

Dr.M (30:28):

I hope the audience here is aware of how honest your earlier answer also was. I love when I ask, what stands in the way of B-Tensor? And you said, you talked about the possibility that it may not work in the way that it is intended to, or this paradigm may not work. And I really appreciate that, because I find that typically, generally, just across the world, people tend to only tell of the bright side. And I also, but you guys don’t do this. And I noticed this also in the question and answer sessions. Internal to the Bitcoin, I’m not Bitcoin, I’m sorry, to the B-Tensor community on Discord, on those sessions, where you guys just say it like it is without any hesitation. And I really appreciate that, honestly.

Ala Shaabana (31:19):

Yeah, go ahead. We’re happy to be honest. Just to kind of take you up a little bit, we’re scientists. We’re not salesmen. So we give you the most scientific answer that we can.

Dr.M (31:31):

You know, there has been, do you guys have a marketing department at all?

Ala Shaabana (31:34):

We have one person. That’s Tak Quillen on the Discord. She’s our entire marketing department, and I think she’s doing a wonderful job. So Tak Quillen, shout out to you.

Dr.M (31:45):

Well done. Amazing, amazing. Very cool. Yeah, you know, it just bewilders me that a few intelligent people, wherever across the world, have developed something that can potentially outpace these multi-billion dollar companies with thousands of resources, thousands of employees, thousands of developers. But it is taking advantage of the decentralization and the power of the people. And gosh, that is so powerful. You know, the first thing I thought when I read about what B-Tensor is was that, you know, this thought came to me. I thought, oh my god, it’s like Bitcoin, except all of that compute that is just arbitrarily calculating in order to produce really a symbol as good as that symbol is to people and as profound as that idea of decentralization that it brought is, but it’s kind of being wasted. And I thought B-Tensor is like exactly the same thing, except not only is the compute not being wasted, but it’s actually going towards one of the most useful things for humanity. I think people at large are a little bit maybe missing awareness on how much resources are huge companies like Google, OpenAI, Tesla, how much resources these guys are spending to train AI because that is what’s going to get them to resolve those really difficult problems to get to the next level in everything it is that they do. And so on the implications of there being such a neural net potentially in the future, should everything go according to plan, that would be the world’s most intelligent, but it would have been developed from contributions of massive number of people, that’s just incredible. On the mining side, did the miners compete with each other? And I noticed internally to B-Tensor’s community, like there is sometimes like a bit of dismay of when some people are kind of disappointed that maybe whatever model they were using before is not competitive anymore. And I feel like they kind of missed the point that that’s the whole point is like, this is the way that you get B-Tensor to become more intelligent faster is to continuously be trained with better and better models or further and further fine-tuned models.

(34:12):

And so like on every side, so even as a miner, it will force me to get better at what I do or I’ll be phased out. And of course there is incentive to take on that learning. So I kind of appreciate that B-Tensor itself is all about intelligence, but then on other aspects of it, like on serving the models, it’s also forcing me to become better at it. And ultimately really, I’m sure down the line will produce probably some of the most expert ML engineers in the world, because at some point, I imagine this will get so competitive that you really have to know what you’re doing to even be mining it at all.

Ala Shaabana (34:56):

That’s a great point. Yeah, that’s a great point. I think that it’s one of those things that, it has all the potential in the world in the future, right? Everything from producing amazing ML engineers that may not be classically trained. So one of the big revolutions that we saw with YouTube and with selling the internet is the emergence of these kids that are just amazing programmers, right? The classic Silicon Valley story is that you’re a 33-year-old programmer, but your boss is a 17-year-old out of high school, right? Because that’s genuinely the thing that happens, because there’s so many talented people that don’t necessarily, they’re not really, for example, good at school or good at whatever, but they’re really good at doing one specific thing, for example, it could be coding. So kind of the same thing applies in B-Tensor, right? It might, as you said, it might actually incentivize, it will actually incentivize a lot of people to be competitive and some of those people might actually achieve peaks that some of us might not really be able to achieve with our tiny brains compared to them. So it’s going to be really interesting. The other tidbit that’s actually, that I wanted to also highlight about B-Tensor, that’s kind of one of the bigger differences with Bitcoin, besides producing intelligence, is that we have, B-Tensor really is a living, breathing network, right? It’s not just a monotonous mine at this much hash rate, this is what you’ll get out, and then if you keep mining at this hash rate, this is what you’re going to get. Even it might change over time. It’ll change much slower because of the having events and the amount of people that join.

(36:22):

B-Tensor, because there are so many knobs to it, it’s almost like flying a plane. I’m not sure if you’ve had the experience flying in those little Cessnas, but there’s a lot of gears and knobs you have to always have your eye on to kind of keep the plane going at the right rate that you want it to, at the right altitude, at the right pitch, and so on. So it’s the same thing with B-Tensor. There’s a lot of hyperparameter updates. These are all machine learning parameters that we have to tune all the time. Everything from the size of the data set that the miners are pulling in, all the way out to the length of the sentences that they’re learning on, all the way out to even just how the incentive mechanism rewards the top miners and the lowest miners, and so on.

(36:60):

And that’s kind of what makes it kind of more of a living, breathing network, because there’s so many things in it that go into engineering the network on the outside as well as the inside. So you’re not just competitive with your peers, but you are also rewarded fairly, and that’s kind of the harder part that we’re kind of working on right now.

Dr.M (37:18):

Right, right. What are upcoming milestones? I mean, I understand that you guys, as the people working on B-Tensor, are extremely busy on a daily basis, and there’s constant problem solving, I’m sure. But I’m just wondering, from just a general public perspective, sort of what are the milestones that are upcoming, or that you’re working on hitting within whatever time frame?

Ala Shaabana (37:47):

Yeah, there’s quite a few. So the publication that we’re working on is one of the big milestones that we had for ourselves for this year. Another one is listing on, sorry, joining the Polkadot network, really, is what we’re working on as well. So that means plugging our blockchain into the Polkadot’s blockchains. For those of you who are unaware, so Polkadot is really a massive collective. It’s almost a blockchain of blockchains, and it’s its own ecosystem. And with that ecosystem, if you become part of it, you get a lot of security, because it becomes much harder to attack you. You get a lot of transaction validation as well, because you’re depending on their validators and not yours. So everything becomes a lot more cohesive. And so that’s one of the big things that we’re looking into, is to join the Polkadot network. And that’s one of the big engineering challenges we’ve been tackling. This is kind of for this year. It’s up to the end of this year is what’s remaining for us to achieve. For next year, one of the big ones is potentially hitting an image modality, kind of starting to work on that. Working on the subnetworks, which is the one thing we’ve been talking about for a long time, which will enable us to hit the image modality as well as the text modality at the same time, and eventually would enable multimodals. Heck, even multimodal models are also on the milestones for next year, if we can achieve it in time. And then finally, it’s the, we want to hit a point where we want to fail the Howey decentralization test.

(39:09):

For those of you unaware of the Howey decentralization test, for example, Bitcoin fails that test, and that’s why it’s not a security. Same thing for us. We’re classified as security today. We want to remain doing this properly and as legal as we can, so we want to fail that test. So right now, we actually are not decentralized enough, so we actually passed the Howey test, which is not a good thing, which is interesting to say, because sometimes passing is what you want, but in this case, we’re going backwards, so we want to fail that test. And that’s kind of the last one that we want to hit next year.

Dr.M (39:35):

Right, right. Alla, I see questions all the time on Discord of people just asking, where can I buy it, or what exchange do I go to to buy it, and I totally understand why they want to buy it. I’ve been telling, you know, Alla, I’ve been describing BitTensor to my friends and family personally as the world’s very best thing, and really, I’ve never been so interested in really wanting someone, like my sister-in-law, who has no technology background to understand the implications of something like BitTensor succeeding to do what it aims to do, like, in almost a strange way, you know. I’m just wondering if you can just touch on the reason why that exchange, or like, why is it that it’s not so easy to buy Tau, and by that, I just mean you cannot go to Coinbase and buy Tau.

Ala Shaabana (40:42):

Yep, absolutely. So actually, just to your earlier point, the amount of times I’ve shilled BitTensor to my dad is ridiculous. He’s gotten to the point where he’s like, I don’t care anymore, just start mining for me. I don’t even know how to do this.

(40:53):

But, yeah, so to be more specific about the listing, and I think a lot of people have been asking us about this for a long time, as, you know, for folks who are unaware and folks who already know, BitTensor started with a fair launch, right? So basically, it was myself, Konst, the foundation, and kind of the collective of the early folks who found BitTensor through GitHub or through whatever means they were looking for. They were the first miners on the network, right? And then the interesting part is that because we did it this way, this actually put us very high up on the tree of potential projects to pursue for entities like the CFTC or the SEC. So the good thing about this is that we have been doing things the right way. You know, we haven’t gotten greedy. We haven’t done anything silly that has gotten us in legal jeopardy, and we want to keep it that way. So legally speaking, Tau is a security. There’s no way around this right now. We’re not decentralized enough, and if we go about listing it ourselves, that’ll put the entire project into legal limbo, and we don’t want to do that because we want to focus on the AI, frankly. We want to focus on building cool stuff. We want to focus on building this network out and making it as powerful as it can be and writing cool scientific papers and publishing them. We don’t want to focus on lawsuits and dealing with the SEC and CFTC, which, you know, they will eventually review us for these things anyways. We want to do things the right way, and especially for our users as well. We don’t want our coin to be stuck in limbo like a lot of other coins has. So what we’re trying to do is we’re trying to go through the Polkadot network first, which is a decentralized exchange. You know, we’ve been advised that this is actually better said now that we’ve been advised something. We’ve been researching ourselves as well, and that this is actually a more, what’s the word I’m looking for? A more safe route towards decentralization because ultimately, it’s kind of a chicken and egg problem, right? You want to decentralize, but you cannot decentralize without speculating because as soon as we speculate on Tau, then we’re in legal trouble, and we don’t ever want to speculate on that.

(42:57):

We have, we in fact only talk about the AI itself. We don’t even talk about the blockchain aspect because that is the most important part anyways.

(43:03):

Yes. If we go directly to an exchange, we’re speculating right off the bat, and we’re benefiting ourselves right away, and we’re just making a lot of problems for a lot of people that are just completely unnecessary. So that’s why we’re taking our time. We’re making it very slow. It is rather difficult, I completely agree, and I feel everyone’s pain to actually obtain Tau. I get it, but at the same time, this is for the common good, right? This is not a marketing ploy. This is not something we’re playing with or making this on purpose. It’s just better. Once we go on to the Polkadot network, it should be fairly easy for you to kind of exchange dots for Tau. It should be rather easy to do it that way because the dots are actually things that you can purchase very easily because they are actually decentralized, and they’re actually listed legally on a lot of exchanges.

Dr.M (43:44):

Awesome, awesome. Ala, actually, of course, I’m in complete agreement with the way that you were going about this, and actually, the reason I asked that question is just for some of that to come out, and thank you for explaining this. No, absolutely. Thanks for asking. It’s good to clarify. Of course, of course, because the attitude that I see from you guys is quiet. In fact, you could say by the numbers, definitely the opposite of what you typically see in the world of anything having to do with crypto, which is that the founders want to get listed and this and that as quickly as possible, and whereas your attitude, and I hear it in both the TGIFT, which is, thank God, it’s Friday, but on a Thursday to the Discord community, the attitude I see from you guys is how careful you are around all of this because I can feel that you’re wanting BitTensor to be around for a very long time and that you don’t want something as silly as some mistake in the past become a cause of regulatory problems down the line, and I really actually appreciate that. At the same time, I do understand that that’s because I am not an ML developer, I’m not an ML expert by any means, but just being slightly aware of the AI space, when I read what BitTensor is, it was profoundly moving, and so I totally understand why there’s so many people wanting to buy this token, and I wanted the audience to kind of hear that care that you have about making sure that you don’t fall into a pitfall that would be regrettable down the line. So amazing things have been done to get here, of course, by this team. Alla, just tell us a little bit about the team. It’s a very small team, I have heard, I don’t know, but I have heard that you guys are very selective with who you bring in. Tell us a little bit about the team.

Ala Shaabana (45:46):

Yeah, absolutely. We’re a very tiny team, actually. For the longest time, it was just me and Khan, basically coding on BitTensor 24-7. That was a lot of fun for a while, because it’s a little sad that he’s not here, because he would have told you all kinds of stories about me tearing my hairs out, and him just tearing out entire pieces of code and replacing them with something even more interesting, but it was really fun. And then, so the team itself is, actually, we’ve grown large enough to go into sub-teams as well, but we’re about 18 people in total. We’re fairly scattered around the world, most of the concentration’s in Toronto. We have folks out in Western Canada, in Vancouver, we have folks out in Austin, shout out to Kara, who’s on the call, I think Quack is as well. We’ve also got folks out in, obviously, Jacob is in South America, he’s in Peru, and we’ve also got folks out in Europe, and ahead of AI, Taco’s also in South Africa, so we’re fairly scattered. Teams, specifically, is almost divided evenly into AI and blockchain teams. There’s actually two AI teams, there’s one blockchain team. The AI teams are, we are an AI company first, so our main focus is the AI, the blockchain’s actually an engineering thing that we’re working with right now. So, because the innovation comes from AI, all of our R&D and all our focus is on the AI part, whereas the blockchain is, it’s almost a software development team. However, there is some R&D going on in blockchain as well. And, yeah, we’ve also got a very, very tiny marketing department, and that is Taq Kulin, who’s doing a great job, and then there’s myself and Const, and, of course, we’ve got our HR department, who’s doing a wonderful job with selectivity of the people who bring on board. It’s just because, one of the main reasons is that we’re not raising as much money as these massive AI companies. I don’t want to list names, but there’s a lot of people in AI companies, I don’t want to list names, but I think there was one recently that just got $100 million investment, which is, in terms of AI, all that money goes towards training models, right? It goes towards building the city yard and stuff like that, and then eventually it’ll go into either an exit, or like selling the company, or into going into the NASDAQ or something. But we raise very little on purpose, and that’s because we are only working with accredited U.S. investors, and as a result, we are very selective with the team that we have, because we actually don’t want to spend that absurd amount of money on a giant team that’s gonna move to a crawl, right?

(48:08):

Myself and Const have both been parts of companies that expanded too fast, too quickly, and have, as a result, created some garbage code, because they hired anybody off the street, and they ended up being not very good developers as a result, and so that’s why we’re very selective with who we bring on.

Dr.M (48:22):

Yes, yes, and it’s a beautiful thing. You know, what I perceive from around BitTensor and from the development team is just how much care goes into it in every aspect of it that I can think of, and also for the audience, just because I’m sure many people who listen to this will be, may not even know much about AI or even crypto, you know, and you touched on this, Ala, but I really don’t think of BitTensor at all as a crypto project, or don’t put it in the same ranks as, you know, what’s out there in terms of the crypto world, and that it is really using one of the supreme advantages that was brought by blockchain, which is just digital trust, so it’s just using a blockchain to distribute the incentive to its miners and validators, and, I mean, it is, I mean, in my mind, at least, and I’ve been exploring Web3 and AI space for a while now, it is the most appropriate, you know, use case for a blockchain, because oftentimes you see many, many, that really may not need to be, whereas in this case, the blockchain enables something to be transparent and to be trustless, and that it’s just, you know, one of the most and that it’s just, you know, one of the things that has made BitTensor possible.

(49:51):

Ala, anything else that, you know, you want to share about BitTensor, or anything, really, that I haven’t asked you?

Ala Shaabana (49:58):

Ah, no, you kind of nailed it, really, on the head in a lot of ways. I think that one of the things that I wanted to clarify a little bit about us being an AI company working, that basically is using a blockchain as a tool, is that, and I think that’s something that I’d love to be able to, that’s one sore point that I think in Web3 that I think I’d love to have BitTensor hit and solve, and kind of get Web3 moving out of the, kind of the stint that it’s in right now, is that Web3 currently today, and I think that’s maybe me going on a rant, so please feel free to interrupt me. No, please, please. It reminds me a lot of the app stores of iPhones and Androids back in 2006, right? There’s not many crazy, interesting applications yet, and I’m not saying that BitTensor’s the most interesting one by far. There’s a lot of really cool projects. I’m just saying that BitTensor is one of the most competitive ones to hit that sore spot. There wasn’t really a lot of really cool applications that brought app stores to the masses back in 2006, right? The coolest app that I remember back in 2006 that I had, which was like, I think I was still in high school, it was the lighter app, if you remember, called it. It was like a Zippo, you click it, and then you swipe up, and then it lights up a lighter on your phone.

(51:05):

And that was it. Now you have the insanely cool apps on phones that you can have that really have brought everything to the mainstream. Now everything’s in an app, right? Even MVPs are done in apps, right? Most minimal viable products are done in apps. And I think that crypto right now is kind of, or Web3 actually, more specifically, is in that state.

(51:25):

There needs to be some applications that’ll bring the masses to Web3. Now everyone in Web3 understands exactly how important Web3 is, and they understand exactly what the promise is, and they understand what the future’s gonna be in that system. But it’s difficult to bring a layman into that, right? And I think there’s a lot of really cool projects, and I really hope a lot of people see BitTensor in the same level as those other projects that are doing the job of bringing BitTensor to the masses. Sorry, not BitTensor, bringing Web3 to the masses. And that is to really bring a use case that allows everyone in the world to be able to use Web3 in the sense that they’re able all to kind of benefit from it. So one really cool project as an example is Helium. I love Helium. It’s such a cool idea, and it’s such a really cool way to kind of bring Web3 into the masses while solving a really particularly difficult problem.

(52:18):

And I really hope that one day BitTensor will achieve the same thing, getting to the point where people will use BitTensor as an example and say, okay, this project brought, helped bring Web3 to the masses in a lot of ways.

Dr.M (52:30):

Mm-hmm. Yeah, absolutely. Gosh, there is just really sheer excitement when I think about the future. Should everything continue to progress as it has been in a year? We’re coming up on the one hour, and we’ll wrap it in the next couple of minutes, but I just want to point out to people just some of the things that have been mentioned here, like the BitTensor network reaching near or potentially very soon. I’m sure if not by today, it would be very soon. The surpassing GPT-3 capability linguistically and how big of a deal that is because BitTensor has had about a week, short of a year, actually, since operating. And I mean, I think it should be obvious to anyone remotely aware of this space how accelerated the intelligence of this network is in terms of its development. And of course, it’s being enabled by contributions from a massive number of numbers of people, and that’s really a beautiful thing. Someone like myself, who is not one of the foremost, let’s say, or is not at all an ML engineer or is not, I don’t even have a computer science degree, the idea of someone like me contributing in any way to any such systems out there is ridiculous. I would need to, I basically would need to be someone who works for these things, whereas BitTensor is being contributed to by anyone who, I would say at this point, if someone who is technically proficient is able to mine BitTensor, and if you are one of those people, I would highly encourage you to begin learning. And there is definitely a learning curve there as a complete outsider to the realm of AI and how to train models and whatnot, but that, gosh, a lot there is not a more worthwhile thing to me, at least personally, to be involved in. And as my audience can see here, I used to tweet many times a day, I used to host many spaces a day, and it seems like I have pretty much stopped doing everything else for BitTensor, and I’m grateful for it, of course. In fact, one of the people in the audience here originally got Konst to come up in another space and speak, and gosh, I’m so grateful for her, because without her, I wouldn’t, that connection with Konst wouldn’t have happened, and I wouldn’t have interviewed you guys. And you know, Ala, this is the first time in my life that I am kind of going all in, like relative to myself in something, but that should, down the line, God forbid, for any reason, should BitTensor not work out. This is the first time that I would not regret having devoted to it as much as I have, and I think that, at least to me, is a very powerful thing to say about anything.

(55:44):

So, any last words, Ala, before we close on this?

Ala Shaabana (55:46):

I think, honestly, I really appreciate you saying that, and frankly, the really interesting thing about BitTensor is because we’re building on so much knowledge, in the off chance that this doesn’t work out, we put together a lot of scientific research that’s worthy of so much publication, and there’s still going to be more and more building on the field of those giants regardless. The one thing I wanted to add is, if you are technically proficient, you only really need two steps to join BitTensor. One, run a miner. Two, go to Coursera and learn how to build neural networks, and then you’re good to go, you can start building your own stuff.

Dr.M (56:20):

Yes, yes, it’s all open source, and gosh, yeah, and there’s lots of resources out there. Absolutely, guys, you know, I really can’t be more excited about anything. In fact, it’s rare for me to even see an idea that I personally haven’t thought of before or haven’t thought of the possibility of it before, but BitTensor definitely was one of those, to the point that I really was so impressed that I couldn’t believe that it’s real. At first, it seemed like there is no way that there is such a thing already and that it’s operating, and of course, what a tremendous thing to have done, and it just really started with the two of you, Jacob and Ala. Ala, thank you so much for giving us your time. We’ll be here again two Thursdays from now, same time, and if you’re interested in learning more, probably, of course, there is the BitTensor.com website. It’s a very clean website. There is a learn section up there on the top right, but that learn section will get you the very basics of installing BitTensor and things like that if you’re interested in mining or validating, but otherwise, maybe a good place to learn more would also be the Thank God It’s Friday Thursday Q&A session that the team holds with their Discord community. The Discord itself, you can join it on BitTensor.com. There is a Discord link right next to that learn button on the site, and with those for whom I care about, I suppose people in my audience, I have actually come to the point of feeling like it is an injustice on my part not to tell them about BitTensor, and I hope that my audience here hears how tremendous this is, and I hope to also at some point come back and tweet maybe more than not at all.

(58:33):

I love that. Thank you so much for joining us.

Ala Shaabana (58:36):

Of course. Thank you. Thank you so much for having me, and for everyone, the next TGIFT, I think it’s going to be the celebration of our one year, so it’s going to be a very, very special one, and that’s going to be November 3rd, so yes, that is the next one. Celebration of our one year. We’re going to be hosting, again, another AMA. It’s going to be very special, so yes. Please, we’d love to see you. Thank you.

Dr.M (59:01):

Thank you so much. Also, to join that, you can join that on the Discord, and the Discord is open for anyone to join. Thank you so much, everybody. We’ll speak again, and this is the second session on BitTensor, and I feel like just describing BitTensor and what it is deserves more attention than just the one session, so we kind of devoted to that. Later on, we’ll get into more intriguing things, maybe having to do with possibilities of AI down the line and that sort of thing with Ala and Jacob. Ala and Jacob, for the time being, at least this is the plan, they will alternate in this space on a biweekly basis, so next time we should have Ala, I mean Jacob, or on const 1 is the handle.

(59:46):

And thank you so much for joining us. We’ll be here next time. Have a good day, everybody.

Video Description

SUBSCRIBE THE BITTENSOR HUB FOR EVERYTHING BITTENSOR!

This Clip was recorded in Dr.M’s Twitter Space Chat on Oct 27, 2022

Host: Dr.M Speaker: Ala Shaabana

Join Dr.M for deep dives into AI & Bittensor: Every other Thursday | 5 - 6 pm EST

Subscribe to the The Bittensor Hub of everything Bittensor!

Bittensor is an open-source protocol that powers a scalable, globally-distributed, decentralized neural network. The system is designed to incentivize the production of artificial intelligence by training models within a distributed infrastructure and rewarding insight gained through data with a custom digital currency.

Discord: https://discord.gg/Qv3fxVaXyE Website: https://docs.bittensor.com/ Whitepaper: https://drive.google.com/file/d/1VnsobL6lIAAqcA1_Tbm8AYIQscfJV4KU/view Network Map: https://bittensor-explorer-staging.netlify.app/

Socials: Bittensor: https://twitter.com/bittensor_ Dr. M: https://twitter.com/DoctorM_DO Ala Shaabana: https://twitter.com/shibshib89

DONATIONS: 5DHYPY9P8VBSpp6Q8m8hHrq4ME5UTiXfXVFbH32MNV4JcSxH

HASHTAGS: #BITTENSORTAO #BITTENSORMINING #BITTENSORCRYPTO #BITTENSORNETWORK #AI #artificialintelligence

Sponsors

Podscript is a personal project to make podcast transcripts available to everyone for free. Please support this project by following us on Twitter.