|
Building Products at SoundCloud—Part III: Microservices in Scala and Finagle
-
developers.soundcloud.com
-
12 years ago
-
eng
In the first two parts of this series, we talked about how SoundCloud started breaking away from a monolithic Ruby on Rails application into…
|
In November last year, I started developing an infrastructure that would allow us to collect, store, search and retrieve high volume data…
|
|
This article reminds of Rocky IV, when Rocky Balboa prepared to fight on behalf of the U.S. against The USSR’s Ivan Drago by running in the snow, doing push-ups, and lifting bundles of rocks while his opponent used steroids and the most advanced training practices. Yet in the end, Rocky still won: hard, grueling exercises, both then and now, prevailed over the finicky, artisanal workouts that have become so popular as of late. If you have n..
|
|
Building Products at SoundCloud—Part II: Breaking the Monolith
-
developers.soundcloud.com
-
12 years ago
-
eng
In the previous post, we talked about how we enabled our teams to build microservices in Scala, Clojure, and JRuby without coupling them…
|
|
Building Products at SoundCloud—Part II: Breaking the Monolith
-
developers.soundcloud.com
-
12 years ago
-
eng
In the previous post, we talked about how we enabled our teams to build microservices in Scala, Clojure, and JRuby without coupling them…
|
|
I’m too sleepy to do any proper work on Gastropoda today. I was hoping to implement the first food item, but yeah…that’s not happening. This morning I decided to head to the office early to get some work done on a largeish task while I had some distraction-free time. I did get a lot done, but waking up at 4am also meant that I was completely beat by about 1. Now I’m in bed, half asleep, wishing someone would bring me some cheese.
|
|
Earlier this week, AdRoll hosted a group of Hadoop users and developers for their monthly meetup . A member of our data team presented AdRoll’s infrastructure and described how we combine Hadoop with Storm and HBase to achieve both high accuracy and low latency. Here are the slides from that talk.
|
|
A few days ago I asked if those who speak out critically of the attention WWDC garners changed their opinions with last week’s impressive roster of announcements; looks like it changed at least one person’s mind, and rightfully so. Adam’s more favorable opinion of WWDC is not the only noteworthy aspect of this piece, however: I believe he hit the nail squarely on the head when describing Apple’s current and future relationship with Google....
|
|
git is a fantastic choice for tracking the evolution of your code base and to collaborate efficiently with your peers. But what happens when the repository you want to track is really huge? via How to handle big repositories with git – Atlassian Blogs .
|
|
Building Products at SoundCloud —Part I: Dealing with the Monolith
-
developers.soundcloud.com
-
12 years ago
-
eng
Most of SoundCloud’s products are written in Scala, Clojure, or JRuby. This wasn’t always the case. Like other start-ups, SoundCloud was…
|
|
Building Products at SoundCloud —Part I: Dealing with the Monolith
-
developers.soundcloud.com
-
12 years ago
-
eng
Most of SoundCloud’s products are written in Scala, Clojure, or JRuby. This wasn’t always the case. Like other start-ups, SoundCloud was…
|
|
After posting a Rust translation of some k -nearest neighbour code , I got a few comments asking “how would you handle errors if you wanted to?”. This is the perfect chance to briefly demonstrate a few idioms. See my previous post for context and the original code; like with the code in that post, this code compiles with rustc 0.11.0-pre-nightly (e55f64f 2014-06-09 01:11:58 -0700) What type of error handling to use? The “canonic....
|
|
I’ve decided that it’s about time to start recording event histories for snails. I keep going back to the database and checking which event ran on which snail by IDs and such and it’s getting to be a hassle. I’d rather just be able to go to the snail’s profile page and see all the important stuff that’s happened to it. Since this was always meant to be a feature anyway, I figured I may as well do it now.
|
|
In my voyages around the internet, I came across a pair of blog posts which compare the implementation of a k -nearest neighbour ( k -NN) classifier in F# and OCaml. I couldn’t resist writing the code into Rust to see how it fared. Rust is a memory-safe systems language under heavy development; this code compiles with the latest nightly (as of 2014-06-10 12:00 UTC), specifically rustc 0.11.0-pre-nightly (e55f64f 2014-06-09 01:11:....
|
|
A few days ago, shortly after watching the WWDC keynote, I made a short quip on Twitter that went completely unnoticed at the time: “Posit: Apple doesn’t need to build low-end phones: it has Android.” I expected to get some pushback on that statement, and so partly because I did not and partly because I believe it an observation worth exploring, I have decided to expand upon it today. Permalink.
|
|
Alright Internet, we need to talk. You've grown up quite a bit over the last several years, but I've never taken the time to have thetalk with you. You know the one. The safe passwords talk. Your requirements for a minimum length, and a diversity of characters in passwords made many classes of brute force attack more difficult. You've gotten a bit carried away though, it seems. You're actually making the internet a more untrustable place.
|
Query an NTP server from Python This code has been updated to support Python 3. Thanks to @MarqueIV for noticing that it needed updating. I came across a need to query an NTP server to get the current time from a python script. There are some great python modules you can install which will take care of the heavy lifting (one great one is ntplib), but I wanted to be able to do these NTP queries without requiring the host to install 3rd party..
|
|
There has been a lot of coverage recently about a chat-bot “ Eugene Goostman ” passing a variance of the Turing test by convincing 30% of the testers that it was indeed a human by posing as Ukrainian 13 year old boy (to make misspellings and grammar mistakes forgivable I suppose). Naturally I had to give it a try and frankly I can’t see how something like this could convince anyone that its a living human being. I asked a few questions su..
|
|
In the 1980s, email remained a luxury for a small number of larger organizations and universities. Mobile email did not exist. Instead, pagers and cellular telephones were the primary tools for mobile communication. However, in the early 1990s, email rose to prominence with the commercialization of the Internet. Although early smartphones did exist at this time (e.g. IBM’s Simon), email was not considered a mobile technology. But by the ..
|
|
In the 1980s, email remained a luxury for a small number of larger organizations and universities. Mobile email did not exist. Instead, pagers and cellular telephones were the primary tools for mobile communication. However, in the early 1990s, email rose to prominence with the commercialization of the Internet. Although early smartphones did exist at this time (e.g. IBM’s Simon), email was not considered a mobile technology. But by the ..
|
|
Chris DeLeon of HobbyGameDev has written a post all about the importance of starting simple in game development , and of picking projects that make you reach as opposed to scramble in terms of their complexity versus your current skillset.
|
|
WWDC has come and gone, and in its wake Apple has left us a number of great analytical shows discussing the implications of its announcements. Podcasts on this subject are not the only ones you will find here, though: this week, as always, I have something for everyone. Permalink.
|
|
I absolutely loved this book.. It was so emotional and descriptive of the hard times in the 1930’s and 40’s… Definitely in my top…
|
|
Wow, two blog posts in one day. Today I implemented a basic energy system for the snails. I’m trying to keep it moderately realistic for the time being just to implement something quickly, but the idea for the final version is to actually do proper research on exactly how this stuff works and revise the system to match.
|
|
First, a couple project updates. I pre-ordered an Oculus DK2! Come August, Lemma will have Oculus VR support. Huzzah! I also got a sign for our booth at the Midwest Game Developer summit . Got it from eSigns , whom I highly recommend! Insanely cheap, fast, and high quality. I am currently scrambling madly to complete a build for the Indie Megabooth submission deadline on Sunday. Stuff that's done:
|
|
Somehow refactoring turned info rewriting from scratch… So, meet the new shiny mucks app written in AWK language (instead of UNIX Shell). I played with quotes in tmux “send-keys” command, and it turned out that different shells treat quotes in the “read” command very differently, and I found no easy way to overcome it. That’s how AWK came to the rescue. A more mature, yet much simpler language made my code cleaner and shorter.
|
|
Setting up a Google Project In order to use any of the Google APIs for your application, first you need to set up a project in the Google Developer’s Console . Enable all the APIs that you want to use in the APIs tab under APIs and auth. Under the Credentials tab, create a Client ID and Client secret which is used for communication between your application and the API. Enter all the allowed redirect urls in the Redirect URIs field. These a....
|
|
Somehow refactoring turned info rewriting from scratch… So, meet the new shiny mucks app written in AWK language (instead of UNIX Shell). I played with quotes in tmux “send-keys” command, and it turned out that different shells treat quotes in the “read” command very differently, and I found no easy way to overcome it. That’s how AWK came to the rescue. A more mature, yet much simpler language made my code cleaner and shorter.
|
|
Since moving to Sweden I have become a sort of cafe hunter. I roam Södermalm with my laptop, sometimes on my Brompton, scouting out potential sitting spots with WiFi where I can sit and work on my personal projects or read.
|
|
Monday afternoon, at the tail end of a jam-packed and incredible keynote, Craig Federighi introduced yet another “kit” — HomeKit. Continuing an impressive trend present throughout the presentation, this framework makes your entire house an accessory to the iPhone in iOS 8 by allowing those with compatible systems to control everything from security cameras and door locks to lights and, although not explicitly stated, other devices such as t..
|
|
Configuration management is an essential part of securing your infrastructure because it can make sure that it is set up correctly. It is essential that configuration management only enhance security, and not weaken it. Unfortunately, the status-quo of secret management in puppet is pretty poor. In the worst (and most common) case, plain text passwords are found in manifests. If the module author tried harder, sometimes these password str..
|
|
Configuration management is an essential part of securing your infrastructure because it can make sure that it is set up correctly. It is essential that configuration management only enhance security, and not weaken it. Unfortunately, the status-quo of secret management in puppet is pretty poor. In the worst (and most common) case, plain text passwords are found in manifests. If the module author tried harder, sometimes these password str..
|
|
The snail simulation is coming along. Tonight I did some work on racing. The racing formula isn’t anywhere close to final yet, but races do run. Here’s how it works right now:
|
|
As I catch up on missed articles, I came across this noteworthy one from Adam Haworth’s previously-mentioned Sansink, where he talks about fanboyism and WWDC. Worth reading, I believe, especially in the wake of Monday’s spectacular keynote address, because I think his point of view mirrors that of all but the relatively small sect who get genuinely enthused around this time each year: for most of the computer-using world, WWDC is an over-....
|
|
I recently stood up a azure VM to test out some work I’ve been doing around creating my own DNS server. Feeling bold and overconfident I created a machine with a long random name, punched open the firewall for dns and set my custom dns running. Having played with the code till late in the evening I left the box running and went to bed. My machines existence was only known to me, so I just left it there with unfinished code answering dns q..
|
|
Stories like this one from Craig Grannell remind me why I have been so reticent to enter the political landscape in any meaningful capacity up until now, a hesitancy I outlined when linking to Ben Thompson’s recent piece in which he called for greater political involvement for those in the technical professions. Everyone promises peace and to bring the soldiers home, but that will never happen: wars will forever wage on, and someone will ..
|
|
Fascinating short documentary done by Motherboard on Soylent. Since it came out I have been a somewhat vocal proponent of this product, but remained so only in theory: through this site I readily recommended others try Soylent every time I wrote about it, but never actually gave it a shot myself. After seeing this, though, I have finally decided to bite the bullet and order some: I’ll start with the original Soylent, and then, depending on ..
|
|
In a recent post I provided my initial impressions of Docker, which were glowing to put it mildly. After spending more time working with it, I've found that it does still have some additional drawbacks in certain situations just about every situation covered thanks to Vagrant.
|