1
00:00:00,000 --> 00:00:10,000
Welcome to Game of Nodes, a weekly podcast on the cosmos from independent Validator teams.

2
00:00:10,000 --> 00:00:19,280
Hello and welcome to Game of Nodes everybody, the podcast about the cosmos from independent

3
00:00:19,280 --> 00:00:21,040
Validator teams.

4
00:00:21,040 --> 00:00:26,560
On this week's episode we have a few of our members sat out there with probably cosmos

5
00:00:26,560 --> 00:00:28,480
induced fatigue.

6
00:00:28,480 --> 00:00:34,160
So we're expecting some drop-ins a little bit later on.

7
00:00:34,160 --> 00:00:39,200
But we've reached out to fill our panel and we've got Demi here today to join us thanks

8
00:00:39,200 --> 00:00:42,480
for coming Demi on incredibly short notice.

9
00:00:42,480 --> 00:00:47,600
I think we only just realized that a lot of people weren't available this morning for

10
00:00:47,600 --> 00:00:48,960
various reasons.

11
00:00:48,960 --> 00:01:00,360
I think Alec LaFray has been working quite hard on the upgrades and the testing on Scotty.

12
00:01:00,360 --> 00:01:06,080
So I think he's a little bit worn out at the moment and Jaby is deep in data and I don't

13
00:01:06,080 --> 00:01:09,880
think he can pull himself away.

14
00:01:09,880 --> 00:01:16,580
And then we've got Jack has showed up.

15
00:01:16,580 --> 00:01:21,200
So we're just talking about our lack of people today but you know, four is good too.

16
00:01:21,200 --> 00:01:27,640
So I think we can expect potentially block pain to drop in a little bit later.

17
00:01:27,640 --> 00:01:33,680
And I'm not sure who else said they were coming but I'm not sure if I mentioned usurper is

18
00:01:33,680 --> 00:01:37,240
a pretty crook at the moment so he won't be coming.

19
00:01:37,240 --> 00:01:40,640
And yeah, so welcome everybody.

20
00:01:40,640 --> 00:01:46,760
And I think we usually start with a question right, Schultz?

21
00:01:46,760 --> 00:01:51,320
Yeah though, to be honest with you, I can't think of an example of the dumb thing that

22
00:01:51,320 --> 00:01:52,320
happened this week.

23
00:01:52,320 --> 00:01:55,520
Oh, we've got a ton.

24
00:01:55,520 --> 00:02:00,200
I mean, you got a selection, you know, you've got what prompt 19 to work with, you've got

25
00:02:00,200 --> 00:02:05,240
other bits and bobs but I don't know that feels kind of unsatisfying and played out

26
00:02:05,240 --> 00:02:06,480
at this point.

27
00:02:06,480 --> 00:02:12,280
Yeah, I think I don't really like to talk too much anymore about the old prop 16 and

28
00:02:12,280 --> 00:02:16,560
all the subsequent props that stuff's gonna play it play itself out.

29
00:02:16,560 --> 00:02:19,040
The trouble we should just start calling it the troubles.

30
00:02:19,040 --> 00:02:20,560
I like that better.

31
00:02:20,560 --> 00:02:26,640
It is like jam our head in the sand and pretend it doesn't exist until it comes time to vote.

32
00:02:26,640 --> 00:02:29,600
You've got ostriches down there, right?

33
00:02:29,600 --> 00:02:30,600
We have emus.

34
00:02:30,600 --> 00:02:31,600
Emus.

35
00:02:31,600 --> 00:02:34,880
Yeah, they're tasty little fuckers.

36
00:02:34,880 --> 00:02:36,680
I do like emu jerky.

37
00:02:36,680 --> 00:02:40,920
Really, I've never partaken in emu jerky.

38
00:02:40,920 --> 00:02:46,840
I've only seen that they chase people and it doesn't look like fun, so I try to avoid

39
00:02:46,840 --> 00:02:47,840
that.

40
00:02:47,840 --> 00:02:54,320
I'm loosely aware of Australia going to war with the emus, but that's about as far as

41
00:02:54,320 --> 00:02:55,520
my knowledge goes.

42
00:02:55,520 --> 00:03:04,400
Yeah, so that was way back in the day we had like the emu war and I can't remember what

43
00:03:04,400 --> 00:03:09,760
exactly that was about, but there was a lot of emus and they came to town and I think

44
00:03:09,760 --> 00:03:12,800
they either had to all get murdered or hunted out of town.

45
00:03:12,800 --> 00:03:18,000
But we've got like camels, armies out in the outback in Australia.

46
00:03:18,000 --> 00:03:21,360
There's just like just camels man everywhere.

47
00:03:21,360 --> 00:03:25,240
There's just so many of them because they survive really well in the Australian desert.

48
00:03:25,240 --> 00:03:30,720
And in droughts, like when there's no rain for a long time, they all decide to go to

49
00:03:30,720 --> 00:03:36,280
town and that can get quite bad as well.

50
00:03:36,280 --> 00:03:44,000
So Jack, what is the dumbest thing or something notable of that you've seen in the cosmos

51
00:03:44,000 --> 00:03:45,000
this week?

52
00:03:45,000 --> 00:03:46,000
You got anything there?

53
00:03:46,000 --> 00:03:53,360
Oh, I mean, if you see me looking down, I'm trying to submit a proposal to Crescent so

54
00:03:53,360 --> 00:03:56,200
that I can claim the rest of my airdrop.

55
00:03:56,200 --> 00:04:01,160
So if you're asking about real dumb things, that's probably one of them.

56
00:04:01,160 --> 00:04:04,800
Let's see, what else has gone on in the cosmos?

57
00:04:04,800 --> 00:04:08,560
Having to submit the prop was dumb that there was none available to claim your airdrop.

58
00:04:08,560 --> 00:04:12,200
Yeah, I think that's dumb, but also having a governance proposal whose purpose of which

59
00:04:12,200 --> 00:04:19,200
is only to vote to claim airdrop is like, yeah, okay.

60
00:04:19,200 --> 00:04:22,800
It's not the first network to do it.

61
00:04:22,800 --> 00:04:24,280
Stargaze had them numbered.

62
00:04:24,280 --> 00:04:28,200
They had like four or five of them.

63
00:04:28,200 --> 00:04:32,200
Yeah, I think it was like, what was it like?

64
00:04:32,200 --> 00:04:36,240
One of Schiltz, do you remember who was submitting those?

65
00:04:36,240 --> 00:04:37,240
Block Payne?

66
00:04:37,240 --> 00:04:41,600
It was like someone was just like one after the other submitting those bloody things every

67
00:04:41,600 --> 00:04:42,760
time they ran out.

68
00:04:42,760 --> 00:04:47,080
And so now my voting history is terrible on Stargaze because I kept missing them.

69
00:04:47,080 --> 00:04:53,480
Yeah, I don't know about Stargaze, but I know on Lume, on Lum, Whispering of submitted

70
00:04:53,480 --> 00:04:57,800
something like 30 proposals on there to keep the airdrops going.

71
00:04:57,800 --> 00:05:01,600
So it looks absolutely ridiculous on that blockchain right now.

72
00:05:01,600 --> 00:05:02,600
Wow.

73
00:05:02,600 --> 00:05:03,800
That's really funny.

74
00:05:03,800 --> 00:05:07,200
Todd, you seen anything dumb this week, man?

75
00:05:07,200 --> 00:05:10,200
Yeah, I botched a proposal this week.

76
00:05:10,200 --> 00:05:11,200
Cheers.

77
00:05:11,200 --> 00:05:13,680
We've all been there.

78
00:05:13,680 --> 00:05:15,280
Yeah, yeah.

79
00:05:15,280 --> 00:05:20,120
So what were the particulars of this proposal?

80
00:05:20,120 --> 00:05:27,680
We just wanted to raise the minimum proposal fee on Serbis and the description didn't match

81
00:05:27,680 --> 00:05:34,520
the actual parameters because we decided to change it at the last minute.

82
00:05:34,520 --> 00:05:37,680
That was the 100 million versus the 150 million?

83
00:05:37,680 --> 00:05:41,400
Yeah, something, 10 versus 15.

84
00:05:41,400 --> 00:05:51,960
I saw someone else who botched a proposal where I think they meant to say the word million

85
00:05:51,960 --> 00:05:55,960
a few times, but said the word billion instead.

86
00:05:55,960 --> 00:05:57,880
And do you know what I'm talking about, Chelsea?

87
00:05:57,880 --> 00:05:58,880
I can't remember.

88
00:05:58,880 --> 00:06:00,280
Maybe it was Serbis as well or...?

89
00:06:00,280 --> 00:06:01,280
I think that is Serbis.

90
00:06:01,280 --> 00:06:07,200
I think it was for the Osmosis matching external incentives where Serbis meant to put like

91
00:06:07,200 --> 00:06:12,080
1.75 billion total and instead added up to something like, I don't know, like 800 billion

92
00:06:12,080 --> 00:06:13,080
or something.

93
00:06:13,080 --> 00:06:17,080
500, 750 and 1.

94
00:06:17,080 --> 00:06:18,080
Yeah, yeah.

95
00:06:18,080 --> 00:06:19,080
A million.

96
00:06:19,080 --> 00:06:21,320
Yeah, that wasn't me, thankfully.

97
00:06:21,320 --> 00:06:24,120
Todd, you're in good company.

98
00:06:24,120 --> 00:06:29,600
If you go look at the Cosmos Hub, we have a few, I remember Zuckie and I put up a few

99
00:06:29,600 --> 00:06:31,040
failed proposals.

100
00:06:31,040 --> 00:06:37,680
I was just looking at the vote history on MintScan and there's proposal number two,

101
00:06:37,680 --> 00:06:41,960
Adam Transfer Enablement, which failed because we did it wrong.

102
00:06:41,960 --> 00:06:47,200
And then proposal number three passed and then we have a Cosmos Hub three upgrade proposal

103
00:06:47,200 --> 00:06:48,200
on there.

104
00:06:48,200 --> 00:06:54,040
And my absolute favorite one is there's upgrade proposal A, upgrade proposal B, upgrade proposal

105
00:06:54,040 --> 00:07:00,480
D and upgrade proposal E. And if you will notice, only one of those actually worked.

106
00:07:00,480 --> 00:07:06,280
So the first time we tried upgrade proposals in broad, we had tried five times.

107
00:07:06,280 --> 00:07:09,000
I remember that.

108
00:07:09,000 --> 00:07:13,160
I felt like such an idiot.

109
00:07:13,160 --> 00:07:16,760
It was not a fun couple of days of work.

110
00:07:16,760 --> 00:07:19,000
That is a little bit out of control.

111
00:07:19,000 --> 00:07:20,000
Yeah.

112
00:07:20,000 --> 00:07:22,720
Sometimes like, yeah.

113
00:07:22,720 --> 00:07:27,440
So Dimmy, do you have anything dumb that you've seen this week?

114
00:07:27,440 --> 00:07:31,120
No, nothing particular came to my mind.

115
00:07:31,120 --> 00:07:32,120
No?

116
00:07:32,120 --> 00:07:33,120
All right.

117
00:07:33,120 --> 00:07:34,640
I'm usually the same way.

118
00:07:34,640 --> 00:07:36,960
I'm like, oh man, it's like on the spot.

119
00:07:36,960 --> 00:07:38,480
I can't remember anything.

120
00:07:38,480 --> 00:07:39,480
This week.

121
00:07:39,480 --> 00:07:41,440
He's just like living that cool hacker house life.

122
00:07:41,440 --> 00:07:43,000
He's just like, yeah, man.

123
00:07:43,000 --> 00:07:45,120
Deep in it, fish hacker house.

124
00:07:45,120 --> 00:07:46,120
Yeah.

125
00:07:46,120 --> 00:07:48,440
I'm closing it closed here of building.

126
00:07:48,440 --> 00:07:51,440
So I can see the reflection.

127
00:07:51,440 --> 00:07:53,800
Yeah, it's nice.

128
00:07:53,800 --> 00:07:57,520
Every time you post pictures, I'm like, oh, fuck, that looks great.

129
00:07:57,520 --> 00:07:58,520
Yeah, look at that.

130
00:07:58,520 --> 00:08:00,040
Where is it that?

131
00:08:00,040 --> 00:08:02,440
Don't talk to yourself.

132
00:08:02,440 --> 00:08:04,240
Palm Springs.

133
00:08:04,240 --> 00:08:05,240
Okay.

134
00:08:05,240 --> 00:08:06,240
Okay.

135
00:08:06,240 --> 00:08:07,240
Okay.

136
00:08:07,240 --> 00:08:09,520
You found a Palm Springs.

137
00:08:09,520 --> 00:08:12,720
You don't want DJs knocking on your door, man.

138
00:08:12,720 --> 00:08:17,440
Can I see the pool?

139
00:08:17,440 --> 00:08:20,040
I've come to clean the pool.

140
00:08:20,040 --> 00:08:23,480
I, what have I done this week?

141
00:08:23,480 --> 00:08:28,440
So I botched an upgrade this week on dig.

142
00:08:28,440 --> 00:08:29,920
Oh, fun.

143
00:08:29,920 --> 00:08:30,920
Yeah.

144
00:08:30,920 --> 00:08:36,320
Well, so I tend to not do the copy paste.

145
00:08:36,320 --> 00:08:39,720
When I do upgrades, I tend to like type it in.

146
00:08:39,720 --> 00:08:40,720
I don't know.

147
00:08:40,720 --> 00:08:43,000
It's good practice to type things in.

148
00:08:43,000 --> 00:08:45,640
And what happened on dig?

149
00:08:45,640 --> 00:08:51,760
I like I think things are until I did a typo, I think, because I use, I use Cosmovisor on

150
00:08:51,760 --> 00:08:52,760
dig.

151
00:08:52,760 --> 00:08:55,360
And this is one of the reasons I don't like Cosmovisor.

152
00:08:55,360 --> 00:08:59,640
One of the many reasons to not use Cosmovisor.

153
00:08:59,640 --> 00:09:00,640
Yeah.

154
00:09:00,640 --> 00:09:04,240
You get like sniped all the time with Cosmovisor.

155
00:09:04,240 --> 00:09:08,600
But anyway, so I had planned to be out that day and no one else was available.

156
00:09:08,600 --> 00:09:16,040
So I did a, you know, I just had preset up the upgrade, but I had a typo in the bloody

157
00:09:16,040 --> 00:09:18,800
directory.

158
00:09:18,800 --> 00:09:24,040
And so when it came time to do the upgrade, obviously couldn't find the binary and freaked

159
00:09:24,040 --> 00:09:25,040
out.

160
00:09:25,040 --> 00:09:34,120
But I was able to get to a terminal and repair it before we had all the votes to start up.

161
00:09:34,120 --> 00:09:36,640
So I didn't miss a block, but I still botched it.

162
00:09:36,640 --> 00:09:38,440
So that was pretty dumb.

163
00:09:38,440 --> 00:09:39,440
Perfect.

164
00:09:39,440 --> 00:09:40,440
Yeah.

165
00:09:40,440 --> 00:09:42,600
I guess, but it's hard to be there for every upgrade.

166
00:09:42,600 --> 00:09:45,040
It's hard to have someone available for every upgrade.

167
00:09:45,040 --> 00:09:47,760
And lately there's been a lot of upgrades.

168
00:09:47,760 --> 00:09:53,680
It is, this is like one of the reasons why we're trying to hire it strange love is to

169
00:09:53,680 --> 00:09:57,000
make sure that we don't have to have some kind of on call rotation.

170
00:09:57,000 --> 00:10:00,280
Yeah, that's a tough one.

171
00:10:00,280 --> 00:10:01,280
Wait, hold on.

172
00:10:01,280 --> 00:10:02,280
What do you mean?

173
00:10:02,280 --> 00:10:03,920
So you don't have an on call rotation?

174
00:10:03,920 --> 00:10:05,560
How would you get around that?

175
00:10:05,560 --> 00:10:06,560
How would you get around that?

176
00:10:06,560 --> 00:10:09,320
I mean, we do have an on call rotation right now.

177
00:10:09,320 --> 00:10:17,600
There's like two people on the validator, but generally, because like all of the, you

178
00:10:17,600 --> 00:10:25,960
know, all of the time, like the, the slashing times are like 18 hours.

179
00:10:25,960 --> 00:10:28,880
I don't wake myself up if our validators down.

180
00:10:28,880 --> 00:10:31,720
I'll like wake up and I'll be like, Oh, it's been down for eight hours.

181
00:10:31,720 --> 00:10:32,720
Okay.

182
00:10:32,720 --> 00:10:34,400
Time to recover the validator.

183
00:10:34,400 --> 00:10:37,400
So this is why we don't do it on call rotation, Chelsea.

184
00:10:37,400 --> 00:10:38,400
Okay.

185
00:10:38,400 --> 00:10:41,720
I'm picking up what you're putting down.

186
00:10:41,720 --> 00:10:48,920
Believe it or not, when we were doing the parameters for the cosmos hub, everyone was

187
00:10:48,920 --> 00:10:51,800
like, so like, what should we do this miss blocks thing?

188
00:10:51,800 --> 00:10:54,840
Like should this be like a thousand?

189
00:10:54,840 --> 00:10:55,840
Should this be like 10,000?

190
00:10:55,840 --> 00:10:57,840
Should this be like a hundred thousand?

191
00:10:57,840 --> 00:11:02,320
I was like, well, like what's a short amount of time that allows somebody to like sleep

192
00:11:02,320 --> 00:11:05,840
for eight hours and then wake up and have a reasonable amount of time to recover the

193
00:11:05,840 --> 00:11:08,560
validator really somewhere around 18 hours.

194
00:11:08,560 --> 00:11:09,560
We're like, Oh, great.

195
00:11:09,560 --> 00:11:10,560
Cool.

196
00:11:10,560 --> 00:11:11,560
Good.

197
00:11:11,560 --> 00:11:15,680
So that in terms of decentralization of the validator set, that's one of those like

198
00:11:15,680 --> 00:11:19,000
underappreciated really important variables.

199
00:11:19,000 --> 00:11:23,680
Because if it was only an hour, like you would really have to have it on call rotation and

200
00:11:23,680 --> 00:11:25,440
it would be a lot harder.

201
00:11:25,440 --> 00:11:27,840
So yeah, that's that's how we do it.

202
00:11:27,840 --> 00:11:29,840
I shall say.

203
00:11:29,840 --> 00:11:30,840
Yeah.

204
00:11:30,840 --> 00:11:35,920
So it's 28% vote power, right?

205
00:11:35,920 --> 00:11:36,920
Yeah.

206
00:11:36,920 --> 00:11:37,920
What was that?

207
00:11:37,920 --> 00:11:39,640
So your mark phones a little bit quiet today.

208
00:11:39,640 --> 00:11:45,120
I don't know if there's any way to like trying to get out of the brutal light here.

209
00:11:45,120 --> 00:11:47,880
So I'm sitting back from my mic.

210
00:11:47,880 --> 00:11:52,400
You look like you look like one of those dudes like, you know, when they're doing the non

211
00:11:52,400 --> 00:11:57,000
hacker video and they've got like the light on their face.

212
00:11:57,000 --> 00:11:58,000
Yeah.

213
00:11:58,000 --> 00:12:05,480
I mean, it's all fine and good unless you've got a validator with, you know, 28, 30% vote

214
00:12:05,480 --> 00:12:09,680
power and they go down for 10 hours.

215
00:12:09,680 --> 00:12:10,680
That is true.

216
00:12:10,680 --> 00:12:13,360
I wish we had that problem with strange love.

217
00:12:13,360 --> 00:12:16,640
One of these days.

218
00:12:16,640 --> 00:12:20,800
Or yeah, you're doing upgrade and you're sitting there waiting for someone with two people

219
00:12:20,800 --> 00:12:25,200
with 20% to like show up when they wake up or whatever.

220
00:12:25,200 --> 00:12:28,440
Upgrades are the only time where we need to really have like strong coverage.

221
00:12:28,440 --> 00:12:34,000
You know, how do you guys like, do you guys use that we've got all of our discord alerts

222
00:12:34,000 --> 00:12:37,760
filtering into like a common discord channel for that?

223
00:12:37,760 --> 00:12:42,120
But I find they're still like, you kind of have to surf the channels and kind of keep

224
00:12:42,120 --> 00:12:46,640
up with things because you know, it's governance and how are you guys doing that?

225
00:12:46,640 --> 00:12:48,680
Calpa tech is Calpa tech.

226
00:12:48,680 --> 00:12:49,680
It's great.

227
00:12:49,680 --> 00:12:50,680
Calpa tech.

228
00:12:50,680 --> 00:12:51,680
He does a great job.

229
00:12:51,680 --> 00:12:52,680
Shout out, Adriana.

230
00:12:52,680 --> 00:12:57,800
Shout out that I just started using not long ago there.

231
00:12:57,800 --> 00:13:04,680
Discord, not discord, the telegram bot to keep a track of Gov proposals.

232
00:13:04,680 --> 00:13:08,760
So basically every time a Gov proposal comes through, their bot picks it up and spits it

233
00:13:08,760 --> 00:13:10,960
out into your own personal channel.

234
00:13:10,960 --> 00:13:13,880
I mean, telegram pretty handy.

235
00:13:13,880 --> 00:13:18,320
I check them like once every couple of days on all the networks we validate anyway.

236
00:13:18,320 --> 00:13:22,920
That is my personal methodology, but it's good to get a prompt.

237
00:13:22,920 --> 00:13:26,840
You know, if I'm doing some other stuff and I forget about it and then it comes up on

238
00:13:26,840 --> 00:13:31,000
the Calpa tech bot, incredibly handy.

239
00:13:31,000 --> 00:13:32,520
Yeah.

240
00:13:32,520 --> 00:13:33,800
Yeah.

241
00:13:33,800 --> 00:13:39,720
So anyone else got any like tricks?

242
00:13:39,720 --> 00:13:42,960
It's taking to the hardcast on.

243
00:13:42,960 --> 00:13:44,280
Sorry.

244
00:13:44,280 --> 00:13:46,320
I use Zapier.

245
00:13:46,320 --> 00:13:51,000
It's a modular cloud type thing.

246
00:13:51,000 --> 00:13:55,840
And I have that watch the RSS feeds for GitHub releases.

247
00:13:55,840 --> 00:13:56,840
Oh, yeah.

248
00:13:56,840 --> 00:13:57,840
Yeah.

249
00:13:57,840 --> 00:14:04,000
So you actually watch the actual, so you actually watch the GitHub's for their actual tags.

250
00:14:04,000 --> 00:14:05,000
Yeah.

251
00:14:05,000 --> 00:14:08,080
So as soon as there's a new tag, I know.

252
00:14:08,080 --> 00:14:13,680
And I'm like, all right, I've got a Gov proposal going to drop any minute now on whatever chain,

253
00:14:13,680 --> 00:14:14,680
you know.

254
00:14:14,680 --> 00:14:18,240
So you should parlay that into a bot.

255
00:14:18,240 --> 00:14:19,240
Telegram.

256
00:14:19,240 --> 00:14:24,200
I was doing that using panic, which is, which is for telegram.

257
00:14:24,200 --> 00:14:26,160
And I found that to be kind of unreliable.

258
00:14:26,160 --> 00:14:30,200
Sometimes they just post tags, whether they mean for it to be the next release or not.

259
00:14:30,200 --> 00:14:34,200
So it became one of those things where like I started paying attention to it, but there's

260
00:14:34,200 --> 00:14:38,840
so much basically false noise that I almost immediately started knowing you're probably

261
00:14:38,840 --> 00:14:42,200
within the first month of setting it up.

262
00:14:42,200 --> 00:14:46,720
If you're anything like Juno, you would have like a thousand tags before we actually do

263
00:14:46,720 --> 00:14:47,720
a release.

264
00:14:47,720 --> 00:14:48,720
Yeah.

265
00:14:48,720 --> 00:14:56,080
So you're about to say on how do you do it at Statefish?

266
00:14:56,080 --> 00:14:57,240
Oh, yeah.

267
00:14:57,240 --> 00:15:04,440
So we also have the tag tracker, but yeah, also for us, it's not that useful because

268
00:15:04,440 --> 00:15:09,880
sometimes something gets tagged, but it's not time of the upgrade yet.

269
00:15:09,880 --> 00:15:15,920
So we have, of course, a governance tracking system that it's custom integrated.

270
00:15:15,920 --> 00:15:22,920
So we have our business slack and we also use Discord aggregating all the different

271
00:15:22,920 --> 00:15:28,560
announcement channel to single one and keep track of that also on slack.

272
00:15:28,560 --> 00:15:29,560
Yeah.

273
00:15:29,560 --> 00:15:36,600
We use many tools and in general we have basically one dedicated person for like nine networks

274
00:15:36,600 --> 00:15:37,600
or something.

275
00:15:37,600 --> 00:15:42,200
So it's a lot of manual work.

276
00:15:42,200 --> 00:15:44,200
One person per nine networks.

277
00:15:44,200 --> 00:15:45,200
Is that what you said?

278
00:15:45,200 --> 00:15:46,200
No, it's yes.

279
00:15:46,200 --> 00:15:52,400
It's like this only because the protocol team is not that big, but we aim to expand.

280
00:15:52,400 --> 00:15:57,040
How many networks are Statefish validating on?

281
00:15:57,040 --> 00:15:59,320
Yeah, around 30.

282
00:15:59,320 --> 00:16:00,320
30?

283
00:16:00,320 --> 00:16:01,320
Yeah.

284
00:16:01,320 --> 00:16:04,800
Yeah, they do everything too.

285
00:16:04,800 --> 00:16:08,800
They're not going to easy mode all Cosmos Chans.

286
00:16:08,800 --> 00:16:09,800
Yeah.

287
00:16:09,800 --> 00:16:13,280
Yeah, it's a big number.

288
00:16:13,280 --> 00:16:17,080
I know, I think it was Schiltzy.

289
00:16:17,080 --> 00:16:24,480
You guys used to or still have your Discord channel, which also aggregates the announcement

290
00:16:24,480 --> 00:16:26,840
channels of a lot of chains.

291
00:16:26,840 --> 00:16:28,840
Quite handy as well.

292
00:16:28,840 --> 00:16:34,080
Honestly, I think that's a good idea to start, but with how many channels there are in there

293
00:16:34,080 --> 00:16:38,000
now, again, it just has basically completely turned into white noise.

294
00:16:38,000 --> 00:16:40,160
And some of the networks I don't even validate on.

295
00:16:40,160 --> 00:16:45,680
And so again, it's just one of those notifications pop up so frequently that aren't necessarily

296
00:16:45,680 --> 00:16:50,480
worth following that again, it's just something that comes in and then like, mark red, move

297
00:16:50,480 --> 00:16:51,480
on.

298
00:16:51,480 --> 00:16:56,720
Man, there is like so many bloody everyone tags.

299
00:16:56,720 --> 00:17:01,840
It's just so annoying that I think you should just not do that.

300
00:17:01,840 --> 00:17:06,480
If people are subscribed to the channel or haven't muted the channel, then they obviously

301
00:17:06,480 --> 00:17:07,920
want the notifications.

302
00:17:07,920 --> 00:17:10,720
But why haven't it, it's a notification channel.

303
00:17:10,720 --> 00:17:15,320
Don't like add everyone every time you do a bloody notification because then you just

304
00:17:15,320 --> 00:17:22,680
wake up in the morning to a thousand notifications and you're like, oh, God, don't have four

305
00:17:22,680 --> 00:17:24,040
hours to read all these.

306
00:17:24,040 --> 00:17:25,320
Yeah, exactly right.

307
00:17:25,320 --> 00:17:29,280
And the funny thing about that is that Zach from EasyStake and I actually got into it

308
00:17:29,280 --> 00:17:36,000
once over me using Add Everyone because there was a Genesis that was happening and like

309
00:17:36,000 --> 00:17:39,880
half the people got their Genesis wrong because I was just reviewing them and I never know

310
00:17:39,880 --> 00:17:40,880
it was okay.

311
00:17:40,880 --> 00:17:42,000
Everyone like got this wrong.

312
00:17:42,000 --> 00:17:45,360
You basically all need to go in here and fix this.

313
00:17:45,360 --> 00:17:52,160
And now I've like come full circle where now I've told people like, hey, you're at everything

314
00:17:52,160 --> 00:17:55,720
way too much because he came in and like shamed me for it.

315
00:17:55,720 --> 00:17:58,320
I'm like, well, I guess what's the quote?

316
00:17:58,320 --> 00:18:02,880
Live long enough to see yourself become the villain, something that definitely it's very

317
00:18:02,880 --> 00:18:03,880
true, I'll say.

318
00:18:03,880 --> 00:18:04,880
It's a guaranteed.

319
00:18:04,880 --> 00:18:06,520
You have to be.

320
00:18:06,520 --> 00:18:08,960
Yeah, I think you have to be everyone.

321
00:18:08,960 --> 00:18:10,880
Everyone just take this note.

322
00:18:10,880 --> 00:18:14,120
Be sparing with the goddamn everyone or the here.

323
00:18:14,120 --> 00:18:17,680
Well, the here's not so bad because you're probably subscribed to that channel if you

324
00:18:17,680 --> 00:18:18,680
get to hear it.

325
00:18:18,680 --> 00:18:25,680
But just think twice before doing the everyone because it's being annoying.

326
00:18:25,680 --> 00:18:31,640
And I got chipped once for doing the everyone back in the day and I don't think I've had

327
00:18:31,640 --> 00:18:36,800
everyone except for maybe one or two emergencies since.

328
00:18:36,800 --> 00:18:41,400
But yeah, man, it's like it's just too many.

329
00:18:41,400 --> 00:18:50,720
So can kind of parlay into what we were going to talk about today.

330
00:18:50,720 --> 00:18:57,320
And I think we sort of came up with the idea that we might talk about security and people's

331
00:18:57,320 --> 00:19:01,360
various methods of validator security.

332
00:19:01,360 --> 00:19:09,600
So I know that, for example, Shilty and myself or maybe maybe no, sorry, that's Pocotu and

333
00:19:09,600 --> 00:19:14,000
myself sort of subscribe at the moment to about a similar model.

334
00:19:14,000 --> 00:19:16,880
Shilty, I'm not sure you're using it.

335
00:19:16,880 --> 00:19:26,600
And I can almost just assume that Todd is using centuries and and dimmy.

336
00:19:26,600 --> 00:19:33,400
Well, steakfish would have probably centuries for every chain and Jack is a Horcrux man

337
00:19:33,400 --> 00:19:34,680
as far as I can tell.

338
00:19:34,680 --> 00:19:36,520
So we're all good.

339
00:19:36,520 --> 00:19:39,480
We are all Horcrux all day here at Strange Lock.

340
00:19:39,480 --> 00:19:41,480
We are all in on.

341
00:19:41,480 --> 00:19:43,480
Do you guys go?

342
00:19:43,480 --> 00:19:45,640
Oh, no, I've lost you.

343
00:19:45,640 --> 00:19:46,640
There you are.

344
00:19:46,640 --> 00:19:49,240
Yeah, we are all in on threshold validation.

345
00:19:49,240 --> 00:19:50,240
Extremely important.

346
00:19:50,240 --> 00:19:54,160
I can sit here and talk about threshold validation for hours if you want.

347
00:19:54,160 --> 00:20:03,720
So I think starting with me because I'm happy to talk about it, I'm going to put down where

348
00:20:03,720 --> 00:20:06,880
we're at and where we're headed.

349
00:20:06,880 --> 00:20:11,840
And then maybe if we just do a quick round robin of the room and if people are comfortable

350
00:20:11,840 --> 00:20:17,560
to talk about their security, you can give us a quick heads up of your method.

351
00:20:17,560 --> 00:20:27,840
So at King Nerds, when we first started out, we evaluated people's security methodologies.

352
00:20:27,840 --> 00:20:32,160
We talked to a lot of different validators, asked them what they use, found a lot of people

353
00:20:32,160 --> 00:20:34,000
were using centuries.

354
00:20:34,000 --> 00:20:38,800
I found a lot of people probably said they were using centuries and aren't.

355
00:20:38,800 --> 00:20:42,320
And others didn't use centuries.

356
00:20:42,320 --> 00:20:48,880
So we were deciding whether or not to use centuries and trying to think of why we might,

357
00:20:48,880 --> 00:20:52,240
instead of just automatically using them, thinking of why we might use them.

358
00:20:52,240 --> 00:21:00,240
Because as a junior validator, we didn't have a big budget to be spending on extra infrastructure

359
00:21:00,240 --> 00:21:01,680
at the time.

360
00:21:01,680 --> 00:21:09,320
And so we came up with the methodology that we would start out without centuries while

361
00:21:09,320 --> 00:21:16,360
we were like a low risk operation and not really a target of anyone.

362
00:21:16,360 --> 00:21:21,000
And then as we grew, we would look at implementing centuries.

363
00:21:21,000 --> 00:21:32,280
So further to that, I kept poking lots and lots of people and asking the pros and cons.

364
00:21:32,280 --> 00:21:42,560
And in the end, we decided not to run centuries because we couldn't see other than people

365
00:21:42,560 --> 00:21:49,320
trying to protect themselves from the host attacks, we couldn't see a huge benefit for

366
00:21:49,320 --> 00:21:56,840
them if we're already being able to hit all of the blocks and not miss a sign.

367
00:21:56,840 --> 00:22:00,280
So I mean, a lot of people use centuries as a way of getting global coverage so they don't

368
00:22:00,280 --> 00:22:05,520
miss any blocks when they're signing and others are scared of DDoS and others are scared of

369
00:22:05,520 --> 00:22:09,880
people being able to gain access to their box to get keys.

370
00:22:09,880 --> 00:22:13,560
So there's various ways to keep keys off your box.

371
00:22:13,560 --> 00:22:14,560
And there's also a very...

372
00:22:14,560 --> 00:22:18,520
I think that's all of those are great reasons for centuries.

373
00:22:18,520 --> 00:22:20,920
This is why that's the first...

374
00:22:20,920 --> 00:22:25,160
If you're going to run a validator, everyone says don't do it on one node.

375
00:22:25,160 --> 00:22:26,720
And there's a lot of reasons why.

376
00:22:26,720 --> 00:22:31,520
And I think that the recent North Korean attacks on a couple of bridges are great.

377
00:22:31,520 --> 00:22:36,440
It's sort of a harbinger of what's to come in this space.

378
00:22:36,440 --> 00:22:40,240
The internet is one giant PVP game and we're going to see more and more of that.

379
00:22:40,240 --> 00:22:42,800
So don't run with one node.

380
00:22:42,800 --> 00:22:47,360
The easiest and first thing you can do is run multiple centuries.

381
00:22:47,360 --> 00:22:51,160
That provides reliability and extra layer of security.

382
00:22:51,160 --> 00:22:59,920
If you're running a full node for your validator on a node that's completely disconnected from

383
00:22:59,920 --> 00:23:04,280
the internet and only connects to PVP through centuries, that's one barrier.

384
00:23:04,280 --> 00:23:10,280
Somebody can still hone your OS, get into your century and then jump from your century

385
00:23:10,280 --> 00:23:14,440
to the back node once they're in your protected environment.

386
00:23:14,440 --> 00:23:17,080
But that's one...

387
00:23:17,080 --> 00:23:18,720
It's at least a really important one.

388
00:23:18,720 --> 00:23:23,240
And then the next step that you can take after that is either Horcrux or TMKMS.

389
00:23:23,240 --> 00:23:26,440
TMKMS is a great other step.

390
00:23:26,440 --> 00:23:28,960
That gives you a lot more operational flexibility.

391
00:23:28,960 --> 00:23:33,840
Once you divorce the state verification function of validation from the signing function of

392
00:23:33,840 --> 00:23:39,480
validation, you have a lot of flexibility in your deployments that you didn't have before.

393
00:23:39,480 --> 00:23:47,800
TMKMS with this soft sign is definitely the easiest way to do that.

394
00:23:47,800 --> 00:23:50,560
So those are the next steps.

395
00:23:50,560 --> 00:23:57,440
And then you can do all kinds of fancy stuff in terms of making sure that your OSes are

396
00:23:57,440 --> 00:24:01,640
automatically updating on all of your nodes to protect against the common set of attacks

397
00:24:01,640 --> 00:24:05,880
that come through these internet-connected endpoints.

398
00:24:05,880 --> 00:24:10,440
What DME just led for the last Juneau thing, security upgrades for the individual chains

399
00:24:10,440 --> 00:24:12,720
are extremely important.

400
00:24:12,720 --> 00:24:19,480
One thing that we as validators don't talk about a lot is those upgrades bring in new

401
00:24:19,480 --> 00:24:21,720
versions of Go continuously.

402
00:24:21,720 --> 00:24:27,280
They bring in new versions of the underlying HTTP libraries that often fix critical security

403
00:24:27,280 --> 00:24:33,280
vulnerabilities that would allow an attacker to get into your Go process, break out of

404
00:24:33,280 --> 00:24:36,520
that, and then into the host beyond that.

405
00:24:36,520 --> 00:24:45,200
And many of these things, if you go read security reports of serious hacks, it's a chain of

406
00:24:45,200 --> 00:24:49,480
vulnerabilities that are used to dive deeper and deeper into these systems.

407
00:24:49,480 --> 00:24:54,920
So the more and more defense in depth you can put up as a validator operator, the better

408
00:24:54,920 --> 00:24:59,200
chance you have of resisting some of these attacks.

409
00:24:59,200 --> 00:25:02,640
And OrcRox is another one of those defense in depth.

410
00:25:02,640 --> 00:25:07,200
Right now, if you're somebody who's going to steal some OrcRox keys, you're going to

411
00:25:07,200 --> 00:25:09,320
have to get a couple layers into the validator.

412
00:25:09,320 --> 00:25:13,360
You're going to end up with these keys, and then you're going to have to write the cryptography

413
00:25:13,360 --> 00:25:15,520
to reconstitute the whole key.

414
00:25:15,520 --> 00:25:16,520
Yeah.

415
00:25:16,520 --> 00:25:26,000
So I'm a huge fan of where OrcRox is going, and that's ultimately where we want to be

416
00:25:26,000 --> 00:25:27,000
is running OrcRox.

417
00:25:27,000 --> 00:25:31,000
And I think it's probably the closest thing you're going to be able to get to being able

418
00:25:31,000 --> 00:25:41,360
to run like a 5-9 as well because of the redundancy and the ability to sign on any node.

419
00:25:41,360 --> 00:25:49,440
So if you have one go down for any reason, like if the supplier goes down from the internet

420
00:25:49,440 --> 00:25:55,920
connection, one of the big problems is if your validator node is behind in terms of

421
00:25:55,920 --> 00:26:00,160
uptime, one of the biggest problems is if your validator node is on one supplier and

422
00:26:00,160 --> 00:26:06,240
you lose access to it because their internet's gone down, you stop missing blocks, but then

423
00:26:06,240 --> 00:26:12,400
you can't pick up on another spare node or use one of your centuries because if you do

424
00:26:12,400 --> 00:26:17,400
and the other one comes back up and you're not quick enough to catch it, then you can

425
00:26:17,400 --> 00:26:24,120
potentially double sign and brick your node, or brick your validator, permajail your validator.

426
00:26:24,120 --> 00:26:30,480
So pretty tough situation when that happens and you don't have access.

427
00:26:30,480 --> 00:26:39,400
So something, as you mentioned, Jack, TKMS is a way to manage that as well as OrcRox.

428
00:26:39,400 --> 00:26:47,840
So particularly around uptime, OrcRox is awesome and security, OrcRox is awesome, and we can't

429
00:26:47,840 --> 00:26:50,480
wait to get onto it.

430
00:26:50,480 --> 00:26:53,000
We just have to do more testing.

431
00:26:53,000 --> 00:26:58,160
I think making the leap on the first network is scary.

432
00:26:58,160 --> 00:27:02,480
And setting it up for the first time is scary.

433
00:27:02,480 --> 00:27:08,360
The first time I did it, it was the Polychain code.

434
00:27:08,360 --> 00:27:12,680
So it was like, there were zero guardrails on that thing, and it wasn't even really tested.

435
00:27:12,680 --> 00:27:13,920
There were no tests.

436
00:27:13,920 --> 00:27:16,520
The Polychain team handed to me, they were like, this is what we run.

437
00:27:16,520 --> 00:27:18,040
If you want to try it, go for it.

438
00:27:18,040 --> 00:27:19,520
I was like, yeah, I'm cool.

439
00:27:19,520 --> 00:27:20,920
I can run this code.

440
00:27:20,920 --> 00:27:23,160
And I played around with it a little bit.

441
00:27:23,160 --> 00:27:26,600
I spun up a couple local networks with it, and I was like, pretty sure this isn't going

442
00:27:26,600 --> 00:27:27,600
to double sign.

443
00:27:27,600 --> 00:27:31,960
I was like, okay, I'm ready to do this hub upgrade.

444
00:27:31,960 --> 00:27:36,640
And it was the first not for zero height hub upgrade.

445
00:27:36,640 --> 00:27:43,440
So I've got this like, privvalidator state file that my single-dode signer had had prior

446
00:27:43,440 --> 00:27:44,440
to that.

447
00:27:44,440 --> 00:27:50,440
And I'm looking at it, and I'm like, okay.

448
00:27:50,440 --> 00:27:54,800
So what height do I put in these state files on Horcrux?

449
00:27:54,800 --> 00:27:59,040
And there's like two state files on each node on all of these Horcrux clusters.

450
00:27:59,040 --> 00:28:04,480
And I'm sitting here at SSH, did to all the machines, manually setting the height in all

451
00:28:04,480 --> 00:28:08,600
of these state files on Horcrux on this upgrade.

452
00:28:08,600 --> 00:28:15,040
And I turned the thing on, and I'm like, and now I have to wait for 30 minutes for my hub

453
00:28:15,040 --> 00:28:18,480
nodes to come up because it was like a horrible startup.

454
00:28:18,480 --> 00:28:21,920
And so I'm just sitting there, and I'm like, maybe I've already fucked myself.

455
00:28:21,920 --> 00:28:22,920
Like, I don't know.

456
00:28:22,920 --> 00:28:24,920
And I'm not going to know for another 30 minutes.

457
00:28:24,920 --> 00:28:27,080
It was so nail-blinding.

458
00:28:27,080 --> 00:28:29,680
But yeah, it's not nearly as bad as that now.

459
00:28:29,680 --> 00:28:31,680
And like, it's much better understood.

460
00:28:31,680 --> 00:28:33,720
So it's still scary though.

461
00:28:33,720 --> 00:28:37,320
And I think the instructions tell you to go out and take a walk or have a drink after

462
00:28:37,320 --> 00:28:39,200
you finish it.

463
00:28:39,200 --> 00:28:40,760
I do.

464
00:28:40,760 --> 00:28:41,760
Yeah.

465
00:28:41,760 --> 00:28:48,280
I remember, I read those, the initial set of instructions like months ago, certainly

466
00:28:48,280 --> 00:28:49,800
probably had updates since then.

467
00:28:49,800 --> 00:28:57,200
But it seemed like it was a nail-biting experience just reading the instructions.

468
00:28:57,200 --> 00:29:01,040
Do we know who all is using Horcrux?

469
00:29:01,040 --> 00:29:04,720
Or is it just French Club that uses it entirely?

470
00:29:04,720 --> 00:29:05,720
No.

471
00:29:05,720 --> 00:29:07,200
Oh, entirely?

472
00:29:07,200 --> 00:29:13,360
I know there's a few people who have publicly said they've run it on a couple of networks.

473
00:29:13,360 --> 00:29:16,680
Coffee Validator runs that on all their networks.

474
00:29:16,680 --> 00:29:22,280
Larry from Delphi Digital runs it on at least two networks.

475
00:29:22,280 --> 00:29:27,080
There's a couple of Luna validators that I know that run it as well.

476
00:29:27,080 --> 00:29:32,560
Most of those Luna validators are just validating Luna, so for whatever that's worth.

477
00:29:32,560 --> 00:29:37,520
And then I know there's a lot of folks maybe running it on Chihuahua or Cerberus or some

478
00:29:37,520 --> 00:29:41,000
of these smaller chains to test it.

479
00:29:41,000 --> 00:29:48,120
But I'd say there's probably five or six other validators who run it for all their networks.

480
00:29:48,120 --> 00:29:56,120
Yeah, the telegram that you have for that, Jack, there's not a lot.

481
00:29:56,120 --> 00:30:03,160
I think it's probably the same reason why I'm not there yet is the fear.

482
00:30:03,160 --> 00:30:06,920
I think I'm going to jump soon.

483
00:30:06,920 --> 00:30:08,440
I've got the playbooks together.

484
00:30:08,440 --> 00:30:11,160
I've tested it on test nets.

485
00:30:11,160 --> 00:30:13,080
Can you hear me okay now?

486
00:30:13,080 --> 00:30:16,200
You're so little choppy, but I can definitely hear you.

487
00:30:16,200 --> 00:30:17,360
Come join us, Todd.

488
00:30:17,360 --> 00:30:18,360
You come.

489
00:30:18,360 --> 00:30:19,360
It's good.

490
00:30:19,360 --> 00:30:20,600
Yeah, the dark side.

491
00:30:20,600 --> 00:30:21,600
I'm on my way.

492
00:30:21,600 --> 00:30:22,600
Yeah.

493
00:30:22,600 --> 00:30:28,400
So, Todd, jam that microphone up to your face and give us the quick rundown on your setup

494
00:30:28,400 --> 00:30:29,880
if you're comfortable with it.

495
00:30:29,880 --> 00:30:33,040
I assume you're using Centres and go.

496
00:30:33,040 --> 00:30:38,720
So, my standard setup is three Centres in a validator.

497
00:30:38,720 --> 00:30:41,720
Validator in one century in the same data center.

498
00:30:41,720 --> 00:30:46,880
And then the other two on different providers.

499
00:30:46,880 --> 00:30:55,240
So, are you running like a, say, for the private network?

500
00:30:55,240 --> 00:31:00,840
Do you run like a wire guard mesh or do you just use firewalling?

501
00:31:00,840 --> 00:31:06,440
I can actually use a wire guard mesh, but I allow the nodes to talk directly to each

502
00:31:06,440 --> 00:31:07,440
other.

503
00:31:07,440 --> 00:31:10,040
I open up those ports between them.

504
00:31:10,040 --> 00:31:11,760
And, like, what?

505
00:31:11,760 --> 00:31:13,960
At least the IPs or something like that?

506
00:31:13,960 --> 00:31:16,000
Yeah, totally.

507
00:31:16,000 --> 00:31:21,040
Because running it over a private network requires you to turn on that feature.

508
00:31:21,040 --> 00:31:27,240
And I think that might actually be, you know, you don't want to do that.

509
00:31:27,240 --> 00:31:32,200
I don't think you want to publish your private nodes in your address book.

510
00:31:32,200 --> 00:31:34,120
Because then people can see that, right?

511
00:31:34,120 --> 00:31:35,960
Am I wrong there?

512
00:31:35,960 --> 00:31:41,480
I think if you don't have peer exchange on, they don't gossip the private addresses.

513
00:31:41,480 --> 00:31:42,480
I could be wrong.

514
00:31:42,480 --> 00:31:44,720
I mean, I guess you kind of have to be careful with it.

515
00:31:44,720 --> 00:31:49,160
But I had less reliability using wire guard.

516
00:31:49,160 --> 00:31:56,000
So, and, you know, I honestly see Centres a lot more, like you said, how it ensures stability.

517
00:31:56,000 --> 00:32:00,720
So I've got a connection to another data center always.

518
00:32:00,720 --> 00:32:03,640
You know, that's the big thing for me.

519
00:32:03,640 --> 00:32:09,640
And you were mentioning earlier about how you see downtime at a data center.

520
00:32:09,640 --> 00:32:16,080
They fuck up their BGP table and it's offline, right?

521
00:32:16,080 --> 00:32:23,560
One way to help with that is I always configure my validator node to connect out on IPv6.

522
00:32:23,560 --> 00:32:30,680
And Centres connected on IPv4 and that saved my ass on a number of data center outages,

523
00:32:30,680 --> 00:32:38,400
you know, because usually the network goes down for IPv4 or IPv6, but not usually both,

524
00:32:38,400 --> 00:32:40,280
unless it's a hardware thing.

525
00:32:40,280 --> 00:32:42,440
So that helps a lot.

526
00:32:42,440 --> 00:32:44,760
Yeah, other security things.

527
00:32:44,760 --> 00:32:59,080
I think one thing a lot of people screw up is using Docker and UFW, which is the simplified

528
00:32:59,080 --> 00:33:01,560
firewall.

529
00:33:01,560 --> 00:33:07,520
And they don't realize that Docker, anything you publish on Docker gets put in the chain

530
00:33:07,520 --> 00:33:09,680
before UFW rules.

531
00:33:09,680 --> 00:33:14,960
So you've got all these people running nodes out there and containers that have their RPC

532
00:33:14,960 --> 00:33:19,920
exposed and stuff like that.

533
00:33:19,920 --> 00:33:22,360
So I always use IP tables, first of all.

534
00:33:22,360 --> 00:33:27,280
And if the provider has a firewall, I use that as well.

535
00:33:27,280 --> 00:33:29,960
But see what else.

536
00:33:29,960 --> 00:33:37,280
Todd, is your, you've got like some like an analytics type thing as one of your websites

537
00:33:37,280 --> 00:33:44,680
that from memory crawls open 2557 ports, right?

538
00:33:44,680 --> 00:33:46,760
Probably all these people fucking up with Docker.

539
00:33:46,760 --> 00:33:55,120
So what's really strange is when you do a peer exchange, it gives you the RPC endpoint

540
00:33:55,120 --> 00:33:56,440
for the node.

541
00:33:56,440 --> 00:33:59,400
So and then you've got the IP address already.

542
00:33:59,400 --> 00:34:04,200
So if it's all zeros, just I just try it and say, hey, give me your peer list.

543
00:34:04,200 --> 00:34:06,200
And that's how I build those maps.

544
00:34:06,200 --> 00:34:07,800
So yeah.

545
00:34:07,800 --> 00:34:13,080
Probably also got the most robust list of RPCs in all of Cosmos.

546
00:34:13,080 --> 00:34:16,320
Well, yeah, it's all that that has done horribly.

547
00:34:16,320 --> 00:34:17,760
Actually, I don't actually have a list.

548
00:34:17,760 --> 00:34:24,880
It's all stored in memory and then it's dumped out as a gob.

549
00:34:24,880 --> 00:34:26,920
Just a big map.

550
00:34:26,920 --> 00:34:35,400
Let me see what's, I was most of this, I think is the biggest right now with 226 open RPC

551
00:34:35,400 --> 00:34:36,400
nodes.

552
00:34:36,400 --> 00:34:37,400
Wow.

553
00:34:37,400 --> 00:34:40,400
So that's pretty massive.

554
00:34:40,400 --> 00:34:51,080
So I feel like you should share that list with Tom so we can add it to RESTAKE.

555
00:34:51,080 --> 00:34:56,520
Just extra nodes to use in case you have load balance against it.

556
00:34:56,520 --> 00:34:57,520
Yeah.

557
00:34:57,520 --> 00:35:03,160
Imagine that just getting taking everybody's RPC and just load balancing it with a smart

558
00:35:03,160 --> 00:35:05,520
balancer.

559
00:35:05,520 --> 00:35:14,640
And then just, you know, that's the network's big RPC super, super distributed.

560
00:35:14,640 --> 00:35:17,000
It's honestly really not that bad of an idea.

561
00:35:17,000 --> 00:35:21,960
Like if you're doing it for not bad things like RESTAKE, then do it.

562
00:35:21,960 --> 00:35:23,600
Let's just do it.

563
00:35:23,600 --> 00:35:24,600
Testing purposes.

564
00:35:24,600 --> 00:35:25,600
Yeah.

565
00:35:25,600 --> 00:35:26,600
Yeah.

566
00:35:26,600 --> 00:35:30,360
Dimmy, what are they doing over at Stakefish, man?

567
00:35:30,360 --> 00:35:33,440
You guys just century the world?

568
00:35:33,440 --> 00:35:39,520
So I'm not sure how much I can share about the Stakefish infra, but yeah, we follow all

569
00:35:39,520 --> 00:35:40,880
the best practices.

570
00:35:40,880 --> 00:35:47,800
We have remote signers and yeah, of course, the size of the infra, the pencils on the

571
00:35:47,800 --> 00:35:54,240
network, we are going to validate and how much is delegated to the validator itself.

572
00:35:54,240 --> 00:35:59,040
But yeah, we always have a base layer of security that follows all the best practices

573
00:35:59,040 --> 00:36:05,800
and maybe for some more like bigger networks, we also have some extra.

574
00:36:05,800 --> 00:36:12,680
One thing that I'm not sure if anybody else is using is the relays.

575
00:36:12,680 --> 00:36:18,600
I'm not speaking about relays of IBC, but relays between Centres and Validator.

576
00:36:18,600 --> 00:36:23,640
I'm not sure if someone is using that or...

577
00:36:23,640 --> 00:36:25,720
I know a couple people.

578
00:36:25,720 --> 00:36:27,760
What was that question, Dimmy?

579
00:36:27,760 --> 00:36:32,920
So this is the backstop bullshit right here.

580
00:36:32,920 --> 00:36:34,520
It's basically another layer.

581
00:36:34,520 --> 00:36:39,920
So you have the Centres that are connected to the world, the relays that are connected

582
00:36:39,920 --> 00:36:45,960
only between the Centres and the Validator and then the Validator itself.

583
00:36:45,960 --> 00:36:51,520
And it was quite a common thing in the early stage of Cosmosup.

584
00:36:51,520 --> 00:36:55,720
Right now I'm not sure how many people are still doing that and I'm not even sure if

585
00:36:55,720 --> 00:36:58,800
it's useful or not.

586
00:36:58,800 --> 00:37:01,920
I mean, it's another layer of defense.

587
00:37:01,920 --> 00:37:04,440
Yeah, I know Polychain was.

588
00:37:04,440 --> 00:37:06,800
I think the conclusion still does.

589
00:37:06,800 --> 00:37:07,800
Anyway.

590
00:37:07,800 --> 00:37:12,400
And what about Dimmy Validator?

591
00:37:12,400 --> 00:37:13,400
My Validator, yeah.

592
00:37:13,400 --> 00:37:16,560
I also follow the best practices I have.

593
00:37:16,560 --> 00:37:20,400
Centres almost everywhere I validate.

594
00:37:20,400 --> 00:37:22,400
Yeah, except one network.

595
00:37:22,400 --> 00:37:27,160
I'm not going to tell you which, but it's not profitable.

596
00:37:27,160 --> 00:37:30,080
So one node is more than enough.

597
00:37:30,080 --> 00:37:39,800
And so do you try to keep those light with state syncing and stuff like that?

598
00:37:39,800 --> 00:37:42,280
So I always use default pruning.

599
00:37:42,280 --> 00:37:49,400
Like I don't like to have pruning everything or to state sync and not have default history.

600
00:37:49,400 --> 00:37:50,400
I'm not going...

601
00:37:50,400 --> 00:37:51,400
Even on your Centres?

602
00:37:51,400 --> 00:37:52,600
Yeah, yeah, yeah.

603
00:37:52,600 --> 00:37:55,320
I do have default almost everywhere.

604
00:37:55,320 --> 00:38:02,960
And I also have some totally separated nodes that I use maybe as archive or even more

605
00:38:02,960 --> 00:38:03,960
history.

606
00:38:03,960 --> 00:38:10,040
Like, I don't like to run my Validator on some pruned state.

607
00:38:10,040 --> 00:38:14,240
I prefer to have all the history.

608
00:38:14,240 --> 00:38:15,560
So yeah, this is my setup.

609
00:38:15,560 --> 00:38:19,480
All my Centres have default pruning and also the Validator.

610
00:38:19,480 --> 00:38:20,480
Interesting.

611
00:38:20,480 --> 00:38:21,320
Cool.

612
00:38:21,320 --> 00:38:26,440
Yeah, I actually started bringing up a bunch of archive nodes just for taxes.

613
00:38:26,440 --> 00:38:27,440
So...

614
00:38:27,440 --> 00:38:29,000
Yeah, Todd, you and me both.

615
00:38:29,000 --> 00:38:33,480
I maintain full archives separate from my Validator on every network I run on.

616
00:38:33,480 --> 00:38:39,960
And then I'm only keeping the last state of the network on the Centres.

617
00:38:39,960 --> 00:38:42,720
So like real thin.

618
00:38:42,720 --> 00:38:45,840
So with Jack, with your...

619
00:38:45,840 --> 00:38:47,920
So hang on.

620
00:38:47,920 --> 00:38:52,800
Centres, whole crux.

621
00:38:52,800 --> 00:38:55,120
If Centres end whole crux?

622
00:38:55,120 --> 00:38:56,120
Yeah.

623
00:38:56,120 --> 00:38:57,120
So we do...

624
00:38:57,120 --> 00:39:01,680
Our setup, I'll just tell you the whole thing because we've open sourced it and it's publicly

625
00:39:01,680 --> 00:39:05,280
available for anyone who wants to give it a try.

626
00:39:05,280 --> 00:39:12,640
We use Google Compute for our cloud and we use their GKE product, the Google Kubernetes

627
00:39:12,640 --> 00:39:13,640
Engine.

628
00:39:13,640 --> 00:39:16,720
But we have two separate Kubernetes clusters.

629
00:39:16,720 --> 00:39:24,640
So one Kubernetes cluster is three machines underneath it and we do three Centres on top

630
00:39:24,640 --> 00:39:25,640
of that.

631
00:39:25,640 --> 00:39:29,360
So one Centre on each machine.

632
00:39:29,360 --> 00:39:33,480
And those Centres are state synced on networks where it's available.

633
00:39:33,480 --> 00:39:36,440
We use snapshots for networks where it's not.

634
00:39:36,440 --> 00:39:42,440
And those Centres are only keeping the latest state and then throwing out anything new.

635
00:39:42,440 --> 00:39:47,760
There's no indexes, we keep them literally as thin as possible.

636
00:39:47,760 --> 00:39:53,480
And on that cluster, the only port that's available is 26656.

637
00:39:53,480 --> 00:39:56,000
The Kubernetes control plan is private.

638
00:39:56,000 --> 00:39:57,000
Everything else is private.

639
00:39:57,000 --> 00:40:00,320
The only thing that's open to the Internet is 26656.

640
00:40:00,320 --> 00:40:04,960
And then we have, how dare you not to use a Cache?

641
00:40:04,960 --> 00:40:05,960
I know Adam.

642
00:40:05,960 --> 00:40:06,960
I know.

643
00:40:06,960 --> 00:40:08,040
Very not on brand for me.

644
00:40:08,040 --> 00:40:12,320
It's sad for me too.

645
00:40:12,320 --> 00:40:19,560
And then we have the Horcrux cluster and the Horcrux cluster actually reaches out to the

646
00:40:19,560 --> 00:40:24,360
TenderBit instances to connect.

647
00:40:24,360 --> 00:40:28,280
So the Horcrux cluster does not have any open ports.

648
00:40:28,280 --> 00:40:33,120
It's an entirely separate cluster and it's even more locked down than the other cluster.

649
00:40:33,120 --> 00:40:38,160
And in fact, there's a Junk box that's also incredibly locked down.

650
00:40:38,160 --> 00:40:44,200
It is the only box that has access to the Kubernetes API for that cluster.

651
00:40:44,200 --> 00:40:51,320
So a defense in depth type setup.

652
00:40:51,320 --> 00:40:55,240
So you've got your Horcrux swarms.

653
00:40:55,240 --> 00:40:57,400
Let me just see if I've got this understood.

654
00:40:57,400 --> 00:41:01,200
So you've got your Horcrux swarm under a couple of layers.

655
00:41:01,200 --> 00:41:07,760
And then you've got, so when you say Centres, they are all of your nodes that can potentially

656
00:41:07,760 --> 00:41:10,200
sign.

657
00:41:10,200 --> 00:41:12,360
So here's how it works.

658
00:41:12,360 --> 00:41:15,120
All three of those nodes are connected to the P2P network.

659
00:41:15,120 --> 00:41:16,840
They all are validated.

660
00:41:16,840 --> 00:41:18,680
They're all like flagged as validators.

661
00:41:18,680 --> 00:41:21,040
We turn that ProVal thing on.

662
00:41:21,040 --> 00:41:25,000
And what that does is it opens a port on the TenderBit instance.

663
00:41:25,000 --> 00:41:28,000
That port is only available to the Horcrux cluster.

664
00:41:28,000 --> 00:41:33,400
So when TenderBit starts up, it waits for five seconds for something to contact it and

665
00:41:33,400 --> 00:41:35,800
say, hey, I've got a pub key.

666
00:41:35,800 --> 00:41:39,680
And once that happens, then it's connected to the Horcrux cluster in the back.

667
00:41:39,680 --> 00:41:44,200
So the nodes start up, the Horcrux cluster sends the pub key to them, and then they know

668
00:41:44,200 --> 00:41:45,800
what validator they are.

669
00:41:45,800 --> 00:41:50,200
Yeah, so it's just the terminology between them.

670
00:41:50,200 --> 00:41:54,480
So your validators are your Centres, essentially.

671
00:41:54,480 --> 00:42:01,480
Yeah, so it's just, because the terminology just, when you traditionally think about what

672
00:42:01,480 --> 00:42:07,440
a Century and what a validator is, and then you sort of mix together in that sense.

673
00:42:07,440 --> 00:42:17,000
So you're not having your cluster of nodes that can sign with the pub key from your

674
00:42:17,000 --> 00:42:18,640
Horcrux swarm.

675
00:42:18,640 --> 00:42:24,320
Then behind another layer of Centres, it's the same thing where all of them can sign.

676
00:42:24,320 --> 00:42:25,320
Yeah.

677
00:42:25,320 --> 00:42:33,040
So you could do another layer like Demi was talking about with the relay nodes.

678
00:42:33,040 --> 00:42:35,200
There's a number of things you can do there.

679
00:42:35,200 --> 00:42:43,920
What my thought on the scaling of this is to do, to auto scale the cluster.

680
00:42:43,920 --> 00:42:50,960
Also there's a change coming to the TenderBit prevale, where instead of the TenderBit instance

681
00:42:50,960 --> 00:42:58,440
opening up a server that then waits for the pub key request, the prevale server is going

682
00:42:58,440 --> 00:43:03,600
to have to open up a server and TenderBit is going to reach out to that via the GRPC

683
00:43:03,600 --> 00:43:04,600
thing.

684
00:43:04,600 --> 00:43:09,760
So we're going to be doing that GRPC conversion and re-jiggering our architecture a little

685
00:43:09,760 --> 00:43:12,120
bit.

686
00:43:12,120 --> 00:43:14,640
But yeah, that's how that works.

687
00:43:14,640 --> 00:43:16,840
Where was I going with that?

688
00:43:16,840 --> 00:43:18,840
I had a point there.

689
00:43:18,840 --> 00:43:23,000
Well, I have two questions.

690
00:43:23,000 --> 00:43:31,760
One, is that pretty cost effective on Google Cloud to do it that way considering how thin

691
00:43:31,760 --> 00:43:36,280
those nodes are on the Kubernetes cluster?

692
00:43:36,280 --> 00:43:39,920
Is it cost effective?

693
00:43:39,920 --> 00:43:42,560
We are still profitable on all the networks we run on.

694
00:43:42,560 --> 00:43:44,560
I'll say that.

695
00:43:44,560 --> 00:43:46,560
Make five cents.

696
00:43:46,560 --> 00:43:47,560
Yeah.

697
00:43:47,560 --> 00:43:54,320
Well, is it really expensive in comparison to a traditional setup?

698
00:43:54,320 --> 00:43:57,800
No, I don't think it's crazy expensive.

699
00:43:57,800 --> 00:44:06,360
I ran, I used to run just on raw VMs and the price on the additional cost for the Kubernetes

700
00:44:06,360 --> 00:44:10,120
cluster is minimal for Google.

701
00:44:10,120 --> 00:44:11,760
That's pretty minimal.

702
00:44:11,760 --> 00:44:20,400
But the additional cost is on Osmosis, we have to run the price ECPUs.

703
00:44:20,400 --> 00:44:26,400
If you want to run the expensive desks, then that's a 10x increase in the monthly cost.

704
00:44:26,400 --> 00:44:32,960
I think we're running, I'm going to quote you a number and I feel like it's off by $100

705
00:44:32,960 --> 00:44:40,440
or so, but I think it's about $650 a month for the three node setup and standard config

706
00:44:40,440 --> 00:44:41,440
for us.

707
00:44:41,440 --> 00:44:50,040
That's in my mind what the number is for what a validator costs a month for us to run one.

708
00:44:50,040 --> 00:44:54,920
The second question is, do you feel like a DGEN not having history on your validators?

709
00:44:54,920 --> 00:44:55,920
No.

710
00:44:55,920 --> 00:44:56,920
No?

711
00:44:56,920 --> 00:44:57,920
You're good?

712
00:44:57,920 --> 00:44:58,920
Yeah.

713
00:44:58,920 --> 00:45:04,640
Obviously, you keep your own separate archive nodes.

714
00:45:04,640 --> 00:45:07,360
Is that on junk hardware or is that?

715
00:45:07,360 --> 00:45:10,160
Yeah, they're generally run on cheap hardware.

716
00:45:10,160 --> 00:45:13,760
Again, some of these, you do have to run them on fairly expensive hardware just to keep

717
00:45:13,760 --> 00:45:16,880
up with chain tips.

718
00:45:16,880 --> 00:45:19,240
Those archives are a fucking beast.

719
00:45:19,240 --> 00:45:22,400
We actually run three different classes of nodes.

720
00:45:22,400 --> 00:45:28,800
We have the super thin centuries and then we have relayer nodes.

721
00:45:28,800 --> 00:45:33,400
Those relayer nodes keep an unbonding period of data.

722
00:45:33,400 --> 00:45:39,640
Then we have the full archives for, as Todd mentioned, paying the federal government.

723
00:45:39,640 --> 00:45:42,280
That's kind of our setup.

724
00:45:42,280 --> 00:45:48,880
Shilty, man, we haven't got to you.

725
00:45:48,880 --> 00:45:52,880
While I appreciate the conversation we're having here, well, I guess let me go through

726
00:45:52,880 --> 00:45:57,560
first what we do and then I'll do my caveat.

727
00:45:57,560 --> 00:45:59,440
Yeah, go ahead.

728
00:45:59,440 --> 00:46:01,600
No, no, no.

729
00:46:01,600 --> 00:46:03,520
I guess I'll just jump right into it.

730
00:46:03,520 --> 00:46:04,600
Basically, we're talking about centuries.

731
00:46:04,600 --> 00:46:07,080
We're talking about validators specific best practices.

732
00:46:07,080 --> 00:46:14,240
One thing we haven't really touched on is in most docs you see on anything, you see

733
00:46:14,240 --> 00:46:19,040
that you're supposed to use the root user for your validator.

734
00:46:19,040 --> 00:46:23,640
The reality is I don't think many of us that are experienced use root, but a lot of the

735
00:46:23,640 --> 00:46:28,760
docs assume you use root and all docs are written for it.

736
00:46:28,760 --> 00:46:30,640
In order to note, go ahead.

737
00:46:30,640 --> 00:46:32,800
No docs I've ever written have.

738
00:46:32,800 --> 00:46:34,400
All right.

739
00:46:34,400 --> 00:46:35,400
That's fair.

740
00:46:35,400 --> 00:46:40,080
Most that you encounter do because if you then add in the complexity of saying, okay,

741
00:46:40,080 --> 00:46:43,800
you should have this user created, then it just adds one more layer of complexity to

742
00:46:43,800 --> 00:46:45,800
creating the docs.

743
00:46:45,800 --> 00:46:51,640
I think it's tough because a lot of the validators that spin up are still using root and they

744
00:46:51,640 --> 00:46:57,960
don't know to actually turn on even their UFW settings as the very base level of protection.

745
00:46:57,960 --> 00:47:03,360
There's just two things that I can guarantee you fewer than 50% of validators use.

746
00:47:03,360 --> 00:47:09,240
One, users, or you could even go so far as doing it so that users, the specific user

747
00:47:09,240 --> 00:47:16,320
doesn't have pseudo access because you could have like, you could have it such that a user

748
00:47:16,320 --> 00:47:22,000
doesn't have pseudo access, it does all the files and then you go into root or some other

749
00:47:22,000 --> 00:47:26,600
user that has pseudo access to that starts up the machine itself.

750
00:47:26,600 --> 00:47:28,560
A lot of people don't do anything like that.

751
00:47:28,560 --> 00:47:29,560
They're just using root.

752
00:47:29,560 --> 00:47:32,360
They have no firewall settings.

753
00:47:32,360 --> 00:47:36,960
Now they probably have the benefit of not having the RPC open because you have to manually

754
00:47:36,960 --> 00:47:42,200
make that change for most networks to actually open it, but otherwise, they're open to the

755
00:47:42,200 --> 00:47:44,240
world full stop.

756
00:47:44,240 --> 00:47:50,040
We're having this conversation about what best practices are we doing, but I don't think

757
00:47:50,040 --> 00:47:55,960
that's the most effective conversation to have to make it the best conversation to have.

758
00:47:55,960 --> 00:47:58,440
We're talking like the advanced stuff.

759
00:47:58,440 --> 00:48:01,080
We should just be talking very low level stuff.

760
00:48:01,080 --> 00:48:04,160
Whenever you start up your machine, one, update everything.

761
00:48:04,160 --> 00:48:05,560
Just update your security things.

762
00:48:05,560 --> 00:48:07,520
We're starting if you need to.

763
00:48:07,520 --> 00:48:08,520
Great.

764
00:48:08,520 --> 00:48:10,080
After that, then you create a new user.

765
00:48:10,080 --> 00:48:13,360
Don't use root and then log into that and do everything from there.

766
00:48:13,360 --> 00:48:14,360
Then add a password.

767
00:48:14,360 --> 00:48:16,400
Turn off password logging in.

768
00:48:16,400 --> 00:48:19,960
Only use SSH keys, blah, blah, blah, blah.

769
00:48:19,960 --> 00:48:23,960
Use UB key in order to lock your SSH key, things like that.

770
00:48:23,960 --> 00:48:29,400
There's this entire conversation to be had that honestly, should be written as a medium

771
00:48:29,400 --> 00:48:30,400
article or something.

772
00:48:30,400 --> 00:48:36,600
People love really invalidator medium articles.

773
00:48:36,600 --> 00:48:44,400
How about like, Shilty, I'd be happy to collaborate with you on that because there should be a

774
00:48:44,400 --> 00:48:51,680
best practice for lower spec.

775
00:48:51,680 --> 00:48:55,800
When I say lower spec, I mean general validators like you're trying to get to because there

776
00:48:55,800 --> 00:49:00,720
is the extreme level like Jack, which is on the tip of the security spear.

777
00:49:00,720 --> 00:49:08,600
There's security professionals like Todd who just couldn't possibly see himself not doing

778
00:49:08,600 --> 00:49:10,200
things a certain way.

779
00:49:10,200 --> 00:49:13,200
But then there's a lot of entry level validators.

780
00:49:13,200 --> 00:49:19,480
There's a lot of mid tier validators that either don't have the resources to provide

781
00:49:19,480 --> 00:49:27,200
that and or don't have experience to know those best practices in the lower space.

782
00:49:27,200 --> 00:49:32,880
I think you're hitting it right on the nail, right in the head with just so many basic

783
00:49:32,880 --> 00:49:39,440
things you can do to protect yourself from the outside world.

784
00:49:39,440 --> 00:49:48,240
For the most part, even if you've just locked it down to your P2P port and even if you do

785
00:49:48,240 --> 00:49:51,400
have SSH open, which is not a good idea.

786
00:49:51,400 --> 00:49:56,440
But when was the last time there was a zero day on SSH if you've configured it properly

787
00:49:56,440 --> 00:50:00,680
with no root and certificates?

788
00:50:00,680 --> 00:50:05,800
There are a few things you can do to get yourself set up more protected than just you're lowering

789
00:50:05,800 --> 00:50:09,920
in with root and copy pasting yourself to glory.

790
00:50:09,920 --> 00:50:10,920
Exactly right.

791
00:50:10,920 --> 00:50:16,200
And to be perfectly honest, so I come from a software engineering degree and so I'm happy

792
00:50:16,200 --> 00:50:18,480
to go write code, but I didn't know too much about Linux.

793
00:50:18,480 --> 00:50:23,520
I use Linux daily because that's where a lot of code is written, but that doesn't mean

794
00:50:23,520 --> 00:50:25,320
I knew security practices.

795
00:50:25,320 --> 00:50:29,560
So for my first network, I was one of those people who just copy pasted in and then I

796
00:50:29,560 --> 00:50:31,040
was like, well, hold on.

797
00:50:31,040 --> 00:50:32,680
This doesn't seem like it's quite right.

798
00:50:32,680 --> 00:50:35,000
Locking straight into root doesn't seem quite right.

799
00:50:35,000 --> 00:50:40,320
Or if you're using Contabo, I think you're defaulted to Ubuntu.

800
00:50:40,320 --> 00:50:46,160
And then I stumbled upon Jack, I think I have you to blame for the century node architecture

801
00:50:46,160 --> 00:50:47,160
right?

802
00:50:47,160 --> 00:50:50,840
It's your article from 2018, right?

803
00:50:50,840 --> 00:50:54,680
Funny enough, I posted that article.

804
00:50:54,680 --> 00:51:01,280
Greg from fucking Saplapod wrote it.

805
00:51:01,280 --> 00:51:04,840
But yeah, I mean, like I have been the century node advocate.

806
00:51:04,840 --> 00:51:05,840
You can blame me.

807
00:51:05,840 --> 00:51:06,840
Yeah, that's fine.

808
00:51:06,840 --> 00:51:09,400
I didn't realize you weren't the one that wrote this.

809
00:51:09,400 --> 00:51:11,360
Yeah, no, it does.

810
00:51:11,360 --> 00:51:14,560
This is probably the first time I've, well, I think I've said it on podcasts before, but

811
00:51:14,560 --> 00:51:15,560
no one ever remembers.

812
00:51:15,560 --> 00:51:16,560
So I was like, oh, it's Jack.

813
00:51:16,560 --> 00:51:20,400
And I'm like, yeah, they kind of, yeah, I posted it.

814
00:51:20,400 --> 00:51:25,720
Greg didn't want to create a forum account because he didn't want more accounts.

815
00:51:25,720 --> 00:51:28,200
So he was like, Jack, can you post this for me?

816
00:51:28,200 --> 00:51:29,200
But yeah.

817
00:51:29,200 --> 00:51:32,200
Greg, Greg from Akash?

818
00:51:32,200 --> 00:51:37,520
No, Greg from, Greg that nobody knows, the Greg behind Saplapod.

819
00:51:37,520 --> 00:51:40,040
Oh, Saplapod, yeah.

820
00:51:40,040 --> 00:51:41,040
Yeah.

821
00:51:41,040 --> 00:51:45,080
Yeah, I actually, I reached out to Greg to get him on the show here in the next couple

822
00:51:45,080 --> 00:51:46,080
of weeks.

823
00:51:46,080 --> 00:51:47,080
So hopefully we've got that coming up.

824
00:51:47,080 --> 00:51:49,640
Right from Akash that is.

825
00:51:49,640 --> 00:51:50,640
From Akash, yeah.

826
00:51:50,640 --> 00:51:51,640
Yeah.

827
00:51:51,640 --> 00:51:52,640
Quick plug.

828
00:51:52,640 --> 00:51:55,480
Don't know what we're going to talk about, but I hit him up anyway.

829
00:51:55,480 --> 00:51:56,480
I guess Akash.

830
00:51:56,480 --> 00:52:01,360
So anyway, sorry, Schilti.

831
00:52:01,360 --> 00:52:05,880
I don't believe you really finished your story before I butted in and started agreeing with

832
00:52:05,880 --> 00:52:06,880
you.

833
00:52:06,880 --> 00:52:12,520
Yeah, I mean, I guess that's the end of it really is that, you know, there's so many

834
00:52:12,520 --> 00:52:15,560
small things that can be done before we talk about Centuries, before we talk about Quark

835
00:52:15,560 --> 00:52:21,840
Crux, before we talk about TMKMS that would be far more beneficial for, let's say, the

836
00:52:21,840 --> 00:52:27,520
every man validated rather than us hardcore extreme.

837
00:52:27,520 --> 00:52:29,040
I'm on 25 networks.

838
00:52:29,040 --> 00:52:30,040
Extreme.

839
00:52:30,040 --> 00:52:31,040
Extreme.

840
00:52:31,040 --> 00:52:32,040
Exactly right.

841
00:52:32,040 --> 00:52:41,760
They should provide some documentation, like tutorial how to put some base layer security

842
00:52:41,760 --> 00:52:42,760
to your nodes.

843
00:52:42,760 --> 00:52:48,520
Like, yeah, I think I think that is definitely something that we need to work on Schilti.

844
00:52:48,520 --> 00:52:52,000
Yeah, because it is very important.

845
00:52:52,000 --> 00:52:59,800
And I think it probably should be prominent in like, you know, like network documentation,

846
00:52:59,800 --> 00:53:04,640
even if it's just a link to something that validators have collaborated on, just as,

847
00:53:04,640 --> 00:53:09,400
you know, if you're going to, if you're going to YOLO and Dubino validator here is some

848
00:53:09,400 --> 00:53:14,000
baseline security that you should probably observe.

849
00:53:14,000 --> 00:53:22,600
And you know, from a documentation perspective, for a network, for networks, I guess it's

850
00:53:22,600 --> 00:53:27,680
kind of assumed that, well, I've always just assumed, I don't actually know if I'm specifically

851
00:53:27,680 --> 00:53:33,120
right to make a non-privileged user, but I guess I've just always assumed that people

852
00:53:33,120 --> 00:53:38,960
would be using a non-privileged user and that.

853
00:53:38,960 --> 00:53:43,120
And that's why I always write sudo, just assuming that they are.

854
00:53:43,120 --> 00:53:47,560
But I guess that works the same as if you just have root and write sudo, you just don't

855
00:53:47,560 --> 00:53:50,240
have to type a password if you've got passwords.

856
00:53:50,240 --> 00:53:52,120
Yeah, basically right.

857
00:53:52,120 --> 00:53:54,200
You just bypass it.

858
00:53:54,200 --> 00:53:56,200
Yeah.

859
00:53:56,200 --> 00:54:00,880
So kind of important and kind of not addressed anywhere really.

860
00:54:00,880 --> 00:54:01,880
Okay.

861
00:54:01,880 --> 00:54:05,240
So I've signed myself up to begin a medium article.

862
00:54:05,240 --> 00:54:06,240
Got it.

863
00:54:06,240 --> 00:54:07,240
All right.

864
00:54:07,240 --> 00:54:12,440
But like I would happily work through that with you, man.

865
00:54:12,440 --> 00:54:15,840
You know, there might be things that both of us do, there might be things that either

866
00:54:15,840 --> 00:54:18,080
of us do or what have you.

867
00:54:18,080 --> 00:54:20,600
So as baseline security.

868
00:54:20,600 --> 00:54:21,600
I'm sorry.

869
00:54:21,600 --> 00:54:22,600
Go ahead.

870
00:54:22,600 --> 00:54:23,600
No, no, go.

871
00:54:23,600 --> 00:54:24,600
That was done.

872
00:54:24,600 --> 00:54:25,600
I've been talking forever.

873
00:54:25,600 --> 00:54:30,480
I find that automation really helps with this stuff because it's so hard to be consistent

874
00:54:30,480 --> 00:54:31,480
every time.

875
00:54:31,480 --> 00:54:36,560
You know, if you're running somewhere, I think it's somewhere around 20 nodes, it starts

876
00:54:36,560 --> 00:54:43,880
to get to the point where you can't always remember what you've done where and having

877
00:54:43,880 --> 00:54:45,840
automation tools to build that stuff.

878
00:54:45,840 --> 00:54:51,960
Be it, you know, if you're doing in the cloud, something like Terraform or I do a lot of hardware.

879
00:54:51,960 --> 00:54:56,400
So I use Ansible.

880
00:54:56,400 --> 00:55:03,240
And but having the security built into your playbooks is a great way to ensure that it's

881
00:55:03,240 --> 00:55:05,160
always done right.

882
00:55:05,160 --> 00:55:09,880
Because I can't tell you before I started automating stuff how many times I went and

883
00:55:09,880 --> 00:55:14,560
I was like, oh, shit, my RPC is exposed on my validator.

884
00:55:14,560 --> 00:55:21,600
I'm, you know, stuff like that, you know, and it's like, so having a little bit of DevOps

885
00:55:21,600 --> 00:55:24,680
experience helps a lot with the security aspects.

886
00:55:24,680 --> 00:55:31,000
And and every time you make a mistake, you just put it in the playbook and then you won't

887
00:55:31,000 --> 00:55:35,000
make the mistake twice, you know, so infrastructure is code.

888
00:55:35,000 --> 00:55:36,000
Shout out Todd.

889
00:55:36,000 --> 00:55:40,160
Mateo says one of the better validator articles is from Todd.

890
00:55:40,160 --> 00:55:42,200
So take some notes, guys.

891
00:55:42,200 --> 00:55:43,200
Oh, thank you.

892
00:55:43,200 --> 00:55:49,800
Oh, yeah, Todd has single-handedly made our lives of under five much better on multiple

893
00:55:49,800 --> 00:55:53,720
occasions with tender duty, even just today, he was helping me debug something that was

894
00:55:53,720 --> 00:55:58,800
going sideways, but there were many times where that's that saved us.

895
00:55:58,800 --> 00:56:02,600
Tender duty, I'm assuming has saved my life.

896
00:56:02,600 --> 00:56:07,040
It's hilarious because it's it's it's like 200 lines of code.

897
00:56:07,040 --> 00:56:08,840
It's like so simple.

898
00:56:08,840 --> 00:56:17,640
Yeah, man, but like huge shout out to that piece of something I resisted it at first.

899
00:56:17,640 --> 00:56:19,440
I resisted it at first.

900
00:56:19,440 --> 00:56:26,200
And then once I used it, I was like, why did I not get on this train earlier?

901
00:56:26,200 --> 00:56:31,520
It's just the most simple but most effective piece of like.

902
00:56:31,520 --> 00:56:38,480
And it's like the first line of security, not security, but the first line of knowing

903
00:56:38,480 --> 00:56:39,600
whether or not you're assigned.

904
00:56:39,600 --> 00:56:42,960
It's like the base layer fundamental thing.

905
00:56:42,960 --> 00:56:44,200
Did you sign the block?

906
00:56:44,200 --> 00:56:46,760
No, here's a warning.

907
00:56:46,760 --> 00:56:50,840
So it's like, you know, there might be things going wrong with your box beforehand, which

908
00:56:50,840 --> 00:56:56,960
hopefully you'd pick up in other with other, you know, monitoring and alerting.

909
00:56:56,960 --> 00:57:02,120
But if it comes down to it and then that's, you know, everything else fails.

910
00:57:02,120 --> 00:57:03,640
And then that's the last line of defense.

911
00:57:03,640 --> 00:57:04,880
Like, did you sign the thing?

912
00:57:04,880 --> 00:57:09,440
No, here's a warning as a validator, you have one job.

913
00:57:09,440 --> 00:57:10,440
Sign blocks.

914
00:57:10,440 --> 00:57:11,440
Sign block.

915
00:57:11,440 --> 00:57:12,440
Got it.

916
00:57:12,440 --> 00:57:14,760
Got to keep a key online and secure.

917
00:57:14,760 --> 00:57:15,760
That's it.

918
00:57:15,760 --> 00:57:17,760
Yeah, exactly.

919
00:57:17,760 --> 00:57:25,280
Anyway, so what if you guys want to talk amongst yourselves for a second, I have to figure out

920
00:57:25,280 --> 00:57:26,800
where we get amongst yourselves.

921
00:57:26,800 --> 00:57:30,440
Well, I mean, the phrase not here, direct traffic.

922
00:57:30,440 --> 00:57:35,800
So usually when we stall, he keeps the things going, but I can't remember what the things

923
00:57:35,800 --> 00:57:36,800
are.

924
00:57:36,800 --> 00:57:39,760
So I need to go back and look at some fun sound effects.

925
00:57:39,760 --> 00:57:45,320
Can you just like, can you make sound effects happen?

926
00:57:45,320 --> 00:57:53,240
Well last time I made last time I made any sound on here, we got like, what do they call

927
00:57:53,240 --> 00:57:56,360
it on YouTube when you get like, band does.

928
00:57:56,360 --> 00:57:59,800
Yeah, we had to like just DMCA.

929
00:57:59,800 --> 00:58:06,000
Yeah, we had to like just make a big blank bit in there when I pulled up a meme.

930
00:58:06,000 --> 00:58:09,920
At least we didn't get shadow band.

931
00:58:09,920 --> 00:58:10,920
That's a Twitter thing, right?

932
00:58:10,920 --> 00:58:12,000
Yeah, it is.

933
00:58:12,000 --> 00:58:13,080
It's a rumor.

934
00:58:13,080 --> 00:58:14,080
Is it real?

935
00:58:14,080 --> 00:58:15,080
Who knows?

936
00:58:15,080 --> 00:58:16,080
Who knows?

937
00:58:16,080 --> 00:58:19,760
Jay, I was claiming to be shadow band.

938
00:58:19,760 --> 00:58:26,040
A question we've had like from, I'm not sure whether this came through YouTube or Twitter,

939
00:58:26,040 --> 00:58:31,000
and I think we might have almost asked it last week.

940
00:58:31,000 --> 00:58:35,480
So can you run a validator with no coding experience?

941
00:58:35,480 --> 00:58:38,240
Does anyone want to take a stab at that?

942
00:58:38,240 --> 00:58:40,520
Ooh, I could definitely take a stab at that.

943
00:58:40,520 --> 00:58:46,760
Now I do have coding experience, but I've helped numerous validators get started that

944
00:58:46,760 --> 00:58:49,120
I've had no coding experience.

945
00:58:49,120 --> 00:58:54,120
I think really the only thing that's required is some level of curiosity.

946
00:58:54,120 --> 00:58:59,680
I think that's the thing that really sets you up for success is like setting up the

947
00:58:59,680 --> 00:59:00,680
validator.

948
00:59:00,680 --> 00:59:05,400
Like at this point, I can set up a new node in let's say five minutes, but even that

949
00:59:05,400 --> 00:59:08,560
feels like a long time for what it can take.

950
00:59:08,560 --> 00:59:10,520
Your first time, it might take 12 hours.

951
00:59:10,520 --> 00:59:13,080
You're running like eight lines that you copy and paste it.

952
00:59:13,080 --> 00:59:15,000
Somehow it takes you 12 hours and you don't know how.

953
00:59:15,000 --> 00:59:20,120
But as long as you're curious and you keep going, it, yeah, you can totally do it without

954
00:59:20,120 --> 00:59:21,120
coding experience.

955
00:59:21,120 --> 00:59:32,440
I am confident that if you jump onto the Uni2 test network on Juno, you can be, you can

956
00:59:32,440 --> 00:59:41,040
not even have bloody experience in a CLI and have a node up within half an hour.

957
00:59:41,040 --> 00:59:45,280
Do you have a link to the Uni2 stuff?

958
00:59:45,280 --> 00:59:48,120
I have one of my lawyer friends that's interested.

959
00:59:48,120 --> 00:59:50,360
Yeah, man.

960
00:59:50,360 --> 00:59:52,960
I'll pull it out while we're talking.

961
00:59:52,960 --> 00:59:56,360
To that end, I want to actually talk about Uni2 a little bit.

962
00:59:56,360 --> 00:59:57,360
Yeah.

963
00:59:57,360 --> 00:59:58,360
Okay.

964
00:59:58,360 --> 00:59:59,360
Yeah.

965
00:59:59,360 --> 01:00:00,360
Sorry for nothing.

966
01:00:00,360 --> 01:00:04,120
Uni2, so that's Juno's testnet.

967
01:00:04,120 --> 01:00:09,040
One of the things that Juno does especially well is the orchestration of their testnet.

968
01:00:09,040 --> 01:00:12,000
And I think that's really what sets Juno up for success.

969
01:00:12,000 --> 01:00:18,640
There was the transition, but I think basically transition between Uni1 and Uni2 was, I think

970
01:00:18,640 --> 01:00:24,440
what created a lot of us is good validators because it was something of a gauntlet of

971
01:00:24,440 --> 01:00:26,800
issues that we found in security holes.

972
01:00:26,800 --> 01:00:28,960
And so we were breaking it daily.

973
01:00:28,960 --> 01:00:32,440
There was six back-to-back genesis transactions.

974
01:00:32,440 --> 01:00:37,080
There were issues that somehow we just kept working through.

975
01:00:37,080 --> 01:00:40,440
It was almost like a boot camp.

976
01:00:40,440 --> 01:00:41,560
It's not like that anymore.

977
01:00:41,560 --> 01:00:45,040
You're not expected to be there 12 hours a day every day anymore.

978
01:00:45,040 --> 01:00:52,120
But during that one transition period, all led by the fray, that made me certainly a

979
01:00:52,120 --> 01:00:55,760
far better validator than it was before that.

980
01:00:55,760 --> 01:01:00,920
That is the best way to get better at validation is to deal with crisis.

981
01:01:00,920 --> 01:01:01,920
Yeah.

982
01:01:01,920 --> 01:01:03,840
Because that is the hard part of validation.

983
01:01:03,840 --> 01:01:08,560
The hard part of validation is not sitting there calmly trying to get your note up and

984
01:01:08,560 --> 01:01:11,200
then you're like, how do I connect to this peer-to-peer network?

985
01:01:11,200 --> 01:01:17,160
Or how long does this thing take to sync, which is always something that newcomers are

986
01:01:17,160 --> 01:01:21,200
shocked by and like, why doesn't my computer just work immediately?

987
01:01:21,200 --> 01:01:22,200
Yeah.

988
01:01:22,200 --> 01:01:26,320
Yeah, I know.

989
01:01:26,320 --> 01:01:32,000
But yeah, it's like when Dimi is desperately trying to find you on Telegram.

990
01:01:32,000 --> 01:01:35,280
And it's like, did you upgrade your binary to version 2.5.6?

991
01:01:35,280 --> 01:01:39,400
Because if you didn't, you're fucked.

992
01:01:39,400 --> 01:01:45,120
That's that real validator shit right there.

993
01:01:45,120 --> 01:01:53,400
Tell me, the days before governance upgrades must have been enjoyable.

994
01:01:53,400 --> 01:01:55,400
Yeah.

995
01:01:55,400 --> 01:01:56,400
Yeah.

996
01:01:56,400 --> 01:01:59,400
It was really funny.

997
01:01:59,400 --> 01:02:01,160
I liked it.

998
01:02:01,160 --> 01:02:02,160
I thought it was fun.

999
01:02:02,160 --> 01:02:03,480
I really enjoyed it.

1000
01:02:03,480 --> 01:02:05,280
I think Dimi did too.

1001
01:02:05,280 --> 01:02:11,080
I imagine that's why Dimi is so good with the spreadsheets for readiness, the readiness

1002
01:02:11,080 --> 01:02:12,080
spreadsheet.

1003
01:02:12,080 --> 01:02:14,080
You can bust out one of those in like 30 seconds.

1004
01:02:14,080 --> 01:02:15,080
Yeah.

1005
01:02:15,080 --> 01:02:16,600
Like a cartel.

1006
01:02:16,600 --> 01:02:20,720
Yeah, that was a throwback.

1007
01:02:20,720 --> 01:02:28,560
I tell you what is good about test networks for anyone who is looking to become a validator

1008
01:02:28,560 --> 01:02:33,080
that watches this or looking for information like how to become a better validator.

1009
01:02:33,080 --> 01:02:41,160
Besides the information that other validators put out to help new people is that if you

1010
01:02:41,160 --> 01:02:44,880
can help it, don't go straight to a mainnet.

1011
01:02:44,880 --> 01:02:51,520
A lot of people yolo into some low value mainnets or even high value mainnets if they can get

1012
01:02:51,520 --> 01:02:52,520
a gathering.

1013
01:02:52,520 --> 01:02:58,440
But recommend just jump in and spend a little while at least in a test net.

1014
01:02:58,440 --> 01:03:01,600
Because you can do things in a test net that you can't do in a mainnet.

1015
01:03:01,600 --> 01:03:03,080
You can attack yourself.

1016
01:03:03,080 --> 01:03:04,160
You can beat yourself up.

1017
01:03:04,160 --> 01:03:05,920
You can ask other people to beat you up.

1018
01:03:05,920 --> 01:03:09,320
You can spam transactions.

1019
01:03:09,320 --> 01:03:14,400
You can try to break things without annoying people.

1020
01:03:14,400 --> 01:03:19,720
There's just so much more stuff you can do to test your own skills and also to benefit

1021
01:03:19,720 --> 01:03:23,640
the network by testing the network.

1022
01:03:23,640 --> 01:03:29,640
If I had any advice for anyone who wants to become a validator, play around in test nets,

1023
01:03:29,640 --> 01:03:30,640
get familiar.

1024
01:03:30,640 --> 01:03:32,640
Just yolo straight into a mainnet.

1025
01:03:32,640 --> 01:03:36,360
Because it's not yourself.

1026
01:03:36,360 --> 01:03:43,440
Even if you are proficient in Linux or whatever you need to run that network, it's not just

1027
01:03:43,440 --> 01:03:44,440
your money.

1028
01:03:44,440 --> 01:03:51,920
If you get jailed or you double sign yourself accidentally, then that's other people's funds

1029
01:03:51,920 --> 01:03:53,480
that you're playing around with as well.

1030
01:03:53,480 --> 01:03:58,720
Yeah, I mean, some of the stuff we went through on uni was fantastic.

1031
01:03:58,720 --> 01:04:04,520
Using the network by messing up state on a bunch of validators, trying to recover from

1032
01:04:04,520 --> 01:04:06,520
a bad block.

1033
01:04:06,520 --> 01:04:13,080
To be fair, we did that on mainnet as well.

1034
01:04:13,080 --> 01:04:14,080
That's a little different.

1035
01:04:14,080 --> 01:04:21,520
Yeah, but I mean, when the ethmost thing was happening, people started seeing the same

1036
01:04:21,520 --> 01:04:24,840
things and I was like, dude, you're screwed.

1037
01:04:24,840 --> 01:04:28,320
Just turn off your node and wait for somebody else to start signing blocks.

1038
01:04:28,320 --> 01:04:32,000
If you don't do that, you're going to double sign.

1039
01:04:32,000 --> 01:04:33,000
Then what happened?

1040
01:04:33,000 --> 01:04:35,440
Everybody double signed.

1041
01:04:35,440 --> 01:04:39,400
I remember you saying that.

1042
01:04:39,400 --> 01:04:40,800
I'm pretty sure you said that in the chat.

1043
01:04:40,800 --> 01:04:43,800
Is there like anyone who's like...

1044
01:04:43,800 --> 01:04:47,080
I'm not a validator there, but I was...

1045
01:04:47,080 --> 01:04:50,960
Some of you guys somewhere else and yeah, somebody mentioned that they were seeing this

1046
01:04:50,960 --> 01:04:53,760
error and it's like, oh, yeah, that's what happened on Juno.

1047
01:04:53,760 --> 01:04:55,760
Just stop your node now.

1048
01:04:55,760 --> 01:04:56,760
Yeah.

1049
01:04:56,760 --> 01:04:58,760
Just stop your node.

1050
01:04:58,760 --> 01:05:01,280
There's a new block.

1051
01:05:01,280 --> 01:05:03,200
When is ethmost going to come back?

1052
01:05:03,200 --> 01:05:04,200
Can we wait?

1053
01:05:04,200 --> 01:05:05,200
Oh, Tuesday.

1054
01:05:05,200 --> 01:05:06,200
Tuesday, 26.

1055
01:05:06,200 --> 01:05:07,200
26.

1056
01:05:07,200 --> 01:05:08,200
Let's go.

1057
01:05:08,200 --> 01:05:09,200
Come on.

1058
01:05:09,200 --> 01:05:10,200
Yeah.

1059
01:05:10,200 --> 01:05:11,200
Want to see it.

1060
01:05:11,200 --> 01:05:15,240
So in there, it was like, yeah, they literally were telling people in the validator chat to

1061
01:05:15,240 --> 01:05:22,040
like re-sync your node after you put down a signature and then explosions happened.

1062
01:05:22,040 --> 01:05:27,240
But yeah, man, I believe that I don't think...

1063
01:05:27,240 --> 01:05:33,840
I don't remember seeing like any public announcement yet that it's coming back on Tuesday, but

1064
01:05:33,840 --> 01:05:41,360
I believe it came from the last community call that it's coming back on the 26th.

1065
01:05:41,360 --> 01:05:42,960
Is what I am gathering.

1066
01:05:42,960 --> 01:05:46,400
I made a tweet about it, but to be fair, I copy it.

1067
01:05:46,400 --> 01:05:50,000
You didn't see any French...

1068
01:05:50,000 --> 01:05:51,000
What's that, Dimitri?

1069
01:05:51,000 --> 01:05:54,440
I didn't see any official announcement as well.

1070
01:05:54,440 --> 01:05:55,560
No.

1071
01:05:55,560 --> 01:05:59,800
So only a few validators have tweeted about it that I've seen and I think it was from the

1072
01:05:59,800 --> 01:06:02,440
last community call.

1073
01:06:02,440 --> 01:06:05,640
I tweeted about it, but I copied it from friends.

1074
01:06:05,640 --> 01:06:10,240
So if it's wrong, blame friends, not me.

1075
01:06:10,240 --> 01:06:12,000
Just retweeting the news, people.

1076
01:06:12,000 --> 01:06:13,000
Retweeting the news.

1077
01:06:13,000 --> 01:06:15,680
That's how quality journalism...

1078
01:06:15,680 --> 01:06:16,680
Yeah.

1079
01:06:16,680 --> 01:06:20,280
That's how it happens now, right?

1080
01:06:20,280 --> 01:06:25,040
Someone makes a tweet and then CNN picks it up.

1081
01:06:25,040 --> 01:06:27,640
So we usually round out with...

1082
01:06:27,640 --> 01:06:30,480
And I think we're at about an hour.

1083
01:06:30,480 --> 01:06:35,160
This is like where we try to start to round out these episodes.

1084
01:06:35,160 --> 01:06:37,840
We usually try to round out with a question again.

1085
01:06:37,840 --> 01:06:43,640
And it is, what are you most excited about this week upcoming?

1086
01:06:43,640 --> 01:06:47,240
So you start with you, Dimitri.

1087
01:06:47,240 --> 01:06:50,680
If you've got something you're excited about, man?

1088
01:06:50,680 --> 01:06:54,600
Yeah, about the raw calculator.

1089
01:06:54,600 --> 01:07:00,280
I hope to be today or tomorrow.

1090
01:07:00,280 --> 01:07:04,800
I think you're not the only person who's excited for you to finish the raw calculator.

1091
01:07:04,800 --> 01:07:08,440
I think a lot of people want to get on their regular address.

1092
01:07:08,440 --> 01:07:13,280
And yeah, I know there is also a lot of people that are unhappy for some decisions that have

1093
01:07:13,280 --> 01:07:14,280
been made.

1094
01:07:14,280 --> 01:07:18,800
I can tell them that there will be a dull.

1095
01:07:18,800 --> 01:07:23,240
So we can do something later as well.

1096
01:07:23,240 --> 01:07:24,240
Potentially do something later.

1097
01:07:24,240 --> 01:07:25,240
Of course.

1098
01:07:25,240 --> 01:07:26,240
They're right checks.

1099
01:07:26,240 --> 01:07:27,240
You can't catch me.

1100
01:07:27,240 --> 01:07:33,240
So like when raw, like Dimitri, do you want to talk snapshots?

1101
01:07:33,240 --> 01:07:34,240
I think it's like...

1102
01:07:34,240 --> 01:07:35,240
I know.

1103
01:07:35,240 --> 01:07:36,240
Please bring it up.

1104
01:07:36,240 --> 01:07:42,040
So Dimitri, is it going to be like...

1105
01:07:42,040 --> 01:07:45,680
I'm just taking snapshots right now.

1106
01:07:45,680 --> 01:07:47,520
I got it on this other screen over here.

1107
01:07:47,520 --> 01:07:48,520
I'm just snapshot.

1108
01:07:48,520 --> 01:07:50,200
Like months of snapshots.

1109
01:07:50,200 --> 01:07:52,920
Guys, just snap shots.

1110
01:07:52,920 --> 01:07:57,360
So is it going to be like just putting your one address and done job?

1111
01:07:57,360 --> 01:08:00,880
Like you can put in any address from all over the cosmos and we'll just reduce it back to

1112
01:08:00,880 --> 01:08:04,600
like the hex and then go from there?

1113
01:08:04,600 --> 01:08:06,400
It will be similar to onata.

1114
01:08:06,400 --> 01:08:10,160
So there will be a page you connect your wallet, you claim.

1115
01:08:10,160 --> 01:08:15,640
If you had multiple wallets, of course, multiple playing to do.

1116
01:08:15,640 --> 01:08:16,640
Awesome.

1117
01:08:16,640 --> 01:08:18,840
Chelsea, man, you got excitement?

1118
01:08:18,840 --> 01:08:19,840
Yeah.

1119
01:08:19,840 --> 01:08:24,320
So I'm actually really excited for the secret network upgrade next week.

1120
01:08:24,320 --> 01:08:27,040
I've been here since secret two and we're a secret four right now and this will be the

1121
01:08:27,040 --> 01:08:30,240
first upgrade that doesn't require a hard fork.

1122
01:08:30,240 --> 01:08:34,520
And that sounds pretty minor, but I'm pretty stoked about it.

1123
01:08:34,520 --> 01:08:42,200
Chelsea, after we did, I'll just say 20 on the order of 20 for the first upgrade where

1124
01:08:42,200 --> 01:08:43,720
it didn't require a hard fork.

1125
01:08:43,720 --> 01:08:46,920
I was just like, this isn't even an upgrade.

1126
01:08:46,920 --> 01:08:47,920
What is this?

1127
01:08:47,920 --> 01:08:48,920
So easy.

1128
01:08:48,920 --> 01:08:49,920
Yeah.

1129
01:08:49,920 --> 01:08:50,920
It feels good.

1130
01:08:50,920 --> 01:08:51,920
It really does.

1131
01:08:51,920 --> 01:08:56,320
Asaf sent me the instructions to review before he posted them and it was like, it was the

1132
01:08:56,320 --> 01:08:57,320
normal six steps.

1133
01:08:57,320 --> 01:09:01,920
Like fetch GitHub, make install, restart node.

1134
01:09:01,920 --> 01:09:02,920
Okay, you're done.

1135
01:09:02,920 --> 01:09:05,520
I was like, oh man, this is great.

1136
01:09:05,520 --> 01:09:06,520
This is fantastic.

1137
01:09:06,520 --> 01:09:07,760
I could do that all day.

1138
01:09:07,760 --> 01:09:09,400
Also shout out Asaf.

1139
01:09:09,400 --> 01:09:14,920
He was the one who finally root cause the contract issue on June.

1140
01:09:14,920 --> 01:09:15,920
Yeah.

1141
01:09:15,920 --> 01:09:17,720
Asaf is absolutely fantastic.

1142
01:09:17,720 --> 01:09:23,240
I interacted with him fairly frequently and every single time he's just fantastic.

1143
01:09:23,240 --> 01:09:24,880
He's such a great person to work with.

1144
01:09:24,880 --> 01:09:26,240
He's awesome.

1145
01:09:26,240 --> 01:09:28,200
We should get him on here one day.

1146
01:09:28,200 --> 01:09:29,200
Shilty.

1147
01:09:29,200 --> 01:09:31,040
Oh, he might be into that.

1148
01:09:31,040 --> 01:09:32,040
That's a good idea.

1149
01:09:32,040 --> 01:09:33,040
Yeah.

1150
01:09:33,040 --> 01:09:34,760
I'd love to have a soft one.

1151
01:09:34,760 --> 01:09:36,240
That'd be fantastic.

1152
01:09:36,240 --> 01:09:37,240
All right.

1153
01:09:37,240 --> 01:09:38,880
Jack, you looking forward to something, man?

1154
01:09:38,880 --> 01:09:39,880
You're not done yet.

1155
01:09:39,880 --> 01:09:40,880
I'm not done yet.

1156
01:09:40,880 --> 01:09:41,880
I'm not done yet.

1157
01:09:41,880 --> 01:09:42,880
Fuck.

1158
01:09:42,880 --> 01:09:46,880
It's the work ever done.

1159
01:09:46,880 --> 01:09:47,880
Am I looking forward to something?

1160
01:09:47,880 --> 01:09:48,880
I don't know.

1161
01:09:48,880 --> 01:09:53,440
We're making chicken casselade tonight.

1162
01:09:53,440 --> 01:09:55,560
So we got some chicken.

1163
01:09:55,560 --> 01:09:57,120
We got some chicken broth.

1164
01:09:57,120 --> 01:09:58,120
It's beans.

1165
01:09:58,120 --> 01:10:00,520
We got some mirepoix.

1166
01:10:00,520 --> 01:10:03,080
You know, just doing the thing.

1167
01:10:03,080 --> 01:10:05,400
So that's what we're working tonight.

1168
01:10:05,400 --> 01:10:06,400
Struggle to remember to eat.

1169
01:10:06,400 --> 01:10:07,400
That sounds delicious.

1170
01:10:07,400 --> 01:10:08,400
Yeah.

1171
01:10:08,400 --> 01:10:10,160
I have not eaten a lot today.

1172
01:10:10,160 --> 01:10:15,080
So I did get some eggs around noon, which is better than nothing, right?

1173
01:10:15,080 --> 01:10:16,800
Like a boiled egg?

1174
01:10:16,800 --> 01:10:17,800
Oh, no.

1175
01:10:17,800 --> 01:10:20,440
I made some sausage and eggs scrambled.

1176
01:10:20,440 --> 01:10:22,240
I prefer mine scrambled personally.

1177
01:10:22,240 --> 01:10:23,240
Cool.

1178
01:10:23,240 --> 01:10:27,080
If I'm feeling really spicy, I'll fry one.

1179
01:10:27,080 --> 01:10:29,080
That's sad.

1180
01:10:29,080 --> 01:10:31,080
Good.

1181
01:10:31,080 --> 01:10:36,920
I'm just glad I'm not sitting around and trying to balance ledgers to figure out how

1182
01:10:36,920 --> 01:10:42,240
much I earned for my greedy uncle.

1183
01:10:42,240 --> 01:10:45,000
You're looking forward to not have to do any tax calculations.

1184
01:10:45,000 --> 01:10:48,520
Well, I've got a lot of work to do yet.

1185
01:10:48,520 --> 01:10:51,920
Still slogging away at it.

1186
01:10:51,920 --> 01:10:57,480
Yeah, but it's nice that I can take a break for a few days.

1187
01:10:57,480 --> 01:10:59,280
I got my estimate in.

1188
01:10:59,280 --> 01:11:00,520
It's probably right.

1189
01:11:00,520 --> 01:11:03,080
I don't think I'm going to jail.

1190
01:11:03,080 --> 01:11:07,840
It's all good.

1191
01:11:07,840 --> 01:11:12,840
Our tax isn't due until the end of like, well, our tax year finishes at the end of June,

1192
01:11:12,840 --> 01:11:15,920
and then it's not due until like May the next year.

1193
01:11:15,920 --> 01:11:19,680
But I just, so May's coming up.

1194
01:11:19,680 --> 01:11:22,600
So I'm still doing tax from a year ago.

1195
01:11:22,600 --> 01:11:29,480
And I put on a crypto bookkeeper to figure it out.

1196
01:11:29,480 --> 01:11:31,520
It's just a completely lost track.

1197
01:11:31,520 --> 01:11:36,760
It was actually a DGEND a bit in Define, and it's just like so hard to figure out that

1198
01:11:36,760 --> 01:11:39,280
I just had to get a bookkeeper, man.

1199
01:11:39,280 --> 01:11:41,480
I just don't have the time.

1200
01:11:41,480 --> 01:11:48,320
But yeah, anyway, so I guess what am I excited about?

1201
01:11:48,320 --> 01:12:03,840
I was talking to Juice Labs yesterday, and I can tell you that I am excited about Juno

1202
01:12:03,840 --> 01:12:07,800
Tools being released whenever they get released.

1203
01:12:07,800 --> 01:12:09,400
Oh, it's ready.

1204
01:12:09,400 --> 01:12:12,200
Yeah, it's ready.

1205
01:12:12,200 --> 01:12:15,760
So when it goes mainnet, I'm excited.

1206
01:12:15,760 --> 01:12:21,600
So I can drop the Bloody King token and people can stop asking me about it because it's got

1207
01:12:21,600 --> 01:12:27,360
the Merkle drop built in and waiting and waiting and waiting and waiting and copying

1208
01:12:27,360 --> 01:12:28,360
so much.

1209
01:12:28,360 --> 01:12:29,360
King shed.

1210
01:12:29,360 --> 01:12:32,080
King shed.

1211
01:12:32,080 --> 01:12:36,720
Just being getting hosed by everyone when King, it's coming in.

1212
01:12:36,720 --> 01:12:38,840
I'm personally really excited about Juno Tools.

1213
01:12:38,840 --> 01:12:42,720
Shout out to Strange Love Engineering team who built the front end there.

1214
01:12:42,720 --> 01:12:43,720
It's been exciting.

1215
01:12:43,720 --> 01:12:44,720
Did they?

1216
01:12:44,720 --> 01:12:47,880
Yeah, sorry, I just gave credit to Juice Labs.

1217
01:12:47,880 --> 01:12:51,480
But can you tell us who's involved other than those guys?

1218
01:12:51,480 --> 01:12:53,320
Juice Labs and then the Strange Love Engineering.

1219
01:12:53,320 --> 01:12:58,560
Juice Labs built the first version of the front end, and then we came in, did some design

1220
01:12:58,560 --> 01:13:04,200
on it, helped them with a logo, built out a lot of the functionality on the front end,

1221
01:13:04,200 --> 01:13:06,240
and they focused on the smart contracts.

1222
01:13:06,240 --> 01:13:13,000
So yeah, Strange Love, if you need front end engineering, holler at your boy.

1223
01:13:13,000 --> 01:13:20,000
So I just walked through that yesterday with Finn, the front end, and he walked me through

1224
01:13:20,000 --> 01:13:21,000
it, man.

1225
01:13:21,000 --> 01:13:22,000
It's pretty slick.

1226
01:13:22,000 --> 01:13:23,640
Grico crushed it.

1227
01:13:23,640 --> 01:13:25,000
He did a great job with it.

1228
01:13:25,000 --> 01:13:29,680
And then my buddy Max did the logo and the visual design.

1229
01:13:29,680 --> 01:13:30,840
It is very nice.

1230
01:13:30,840 --> 01:13:36,000
Actually, the logo is super crisp.

1231
01:13:36,000 --> 01:13:41,120
It's just, I don't know how it is so crisp, but it's crisper than what anything else displays

1232
01:13:41,120 --> 01:13:42,600
on my monitor.

1233
01:13:42,600 --> 01:13:46,520
And then the logo is just like, it almost stands out of the page a little bit.

1234
01:13:46,520 --> 01:13:48,520
I don't know how you did it.

1235
01:13:48,520 --> 01:13:51,200
Max is a great visual designer.

1236
01:13:51,200 --> 01:13:52,200
I love his stuff.

1237
01:13:52,200 --> 01:14:01,440
I've known Max for, I'm about to say the number 15 years, and we went to college together.

1238
01:14:01,440 --> 01:14:05,960
So anyway, yeah, it's been a while.

1239
01:14:05,960 --> 01:14:09,880
Well then, he's knocked it out of the park.

1240
01:14:09,880 --> 01:14:16,080
So I guess that probably brings us to a close for today, unless anyone's got any final words.

1241
01:14:16,080 --> 01:14:20,080
It's some strange coincidence that Pupmos is here, but Javi isn't.

1242
01:14:20,080 --> 01:14:24,880
Yeah, what a coincidence.

1243
01:14:24,880 --> 01:14:27,480
The alter ego coming out.

1244
01:14:27,480 --> 01:14:40,480
Thanks everybody.