|
First of all, apologies for the lack of updates over the past few weeks, I've been busy with a product launch and doing a bit of work-related travel, so I haven't had a chance to sit down and polish off any of posts. However, things are starting to settle down, and I have a bunch of new posts in the pipeline at various levels of readiness, so watch for those over the coming week.
|
|
First of all, apologies for the lack of updates over the past few weeks, I've been busy with a product launch and doing a bit of work-related travel, so I haven't had a chance to sit down and polish off any of posts. However, things are starting to settle down, and I have a bunch of new posts in the pipeline at various levels of readiness, so watch for those over the coming week.
|
|
pAPI developers place plenty of consideration on how to handle scaling. A few techniques and approaches: Application caching, sharding, larger/more servers, background queues, HTTP caching, and Content Delivery Networks (CDNs). A well architected API employs a combination of these solutions to achieve the desired scalability. Of the scaling solutions, very few are actually exposed to the consumers of an API. One strategy in which the end-us....
|
|
My pip installation recently broke after a Mac OS update. $ pip Traceback (most recent call last): File "/usr/local/bin/pip\n", line 5, in
|
|
Petabyte-Scale Data Pipelines with Docker, Luigi and Elastic Spot Instances
-
tech.nextroll.com
-
10 years ago
-
eng
This is the first article in a series that describes how we built a new data-intensive product, AdRoll Prospecting , using an architecture based on Docker containers. We organized a meetup about this topic with Pachyderm.io . You can watch a recording of our meetup talk here: and see the slides here: We will elaborate different aspects of the architecture in upcoming blog posts. The one below will focus on Docker. A Modern Data....
|
|
An ETL import graph is build on logical dependencies of the jobs to each other. So typically a SQL transformation job depends on all the previous jobs that create the tables used in the query. But once there are a certain number of jobs, dependencies often get a bit more complicated and some of them become redundant in the process. A simple example can be seen in the dependency graph from figure, where the three red edges are redundant.
|
|
As a Customer, I Want To See Up-to-Date Costs I've Incurred, Using a Web Application
-
www.databasesandlife.com
-
10 years ago
-
eng
I think it's important, that in a software development company, as all others, customers get transparent up-to-the-minute information about what costs they have incurred. Alas my company can't provide that at the moment, due to our tooling (this is a deficiency, or at least difference of opinion, between me and LiquidPlanner, the tool we use for tracking time.) This is the email I sent to LiquidPlanner support explaining my position: I an..
|
In an attempt to avoid using the Juniper Pulse (Now Pulse Secure) VPN client we tried OpenConnect but found that DNS did not work correctly when connected to the VPN. This bug has now been resolved recently but has not made it’s way into a new build, in fact there have been no releases for 6 months. Luckily the OpenConnect was not too difficult to build from source. Build OpenConnect on OSX Remove old openconnect and install deps brew....
|
|
In an attempt to avoid using the Juniper Pulse (Now Pulse Secure) VPN client we tried OpenConnect but found that DNS did not work correctly when connected to the VPN. This bug has now been resolved recently but has not made it’s way into a new build, in fact there have been no releases for 6 months. Luckily the OpenConnect was not too difficult to build from source. Build OpenConnect on OSX Remove old openconnect and install deps brew....
|
|
There’s a well-understood phenomenon in machine learning called overfitting. The idea is best shown by a graph: overfitting Let me explain. The vertical axis represents the error of a hypothesis. The horizontal axis represents the complexity of the hypothesis. The blue curve represents the error of a machine learning algorithm’s output on its training data, and the red curve represents the generalization of that hypothesis to the real world..
|
|
There’s a well-understood phenomenon in machine learning called overfitting. The idea is best shown by a graph: overfitting Let me explain. The vertical axis represents the error of a hypothesis. The horizontal axis represents the complexity of the hypothesis. The blue curve represents the error of a machine learning algorithm’s output on its training data, and the red curve represents the generalization of that hypothesis to the real world..
|
|
There’s a well-understood phenomenon in machine learning called overfitting. The idea is best shown by a graph: overfitting Let me explain. The vertical axis represents the error of a hypothesis. The horizontal axis represents the complexity of the hypothesis. The blue curve represents the error of a machine learning algorithm’s output on its training data, and the red curve represents the generalization of that hypothesis to the real world..
|
|
RAM is the new disk – and how to measure its performance – Part 2 – Tools
-
tanelpoder.com
-
10 years ago
-
eng
part 1 | part 2 | part 3 ] In the previous article I explained that the main requirement for high-speed in-memory data scanning is column-oriented storage format for in-memory data. SIMD instruction processing is just icing on the cake. Let’s dig deeper. This is a long post, you’ve been warned. Test Environment I will cover full test results in the next article in this series. First, let’s look into the test setup, environment and ....
|
|
RAM is the new disk – and how to measure its performance – Part 2 – Tools
-
tanelpoder.com
-
10 years ago
-
eng
part 1 | part 2 | part 3 ] In the previous article I explained that the main requirement for high-speed in-memory data scanning is column-oriented storage format for in-memory data. SIMD instruction processing is just icing on the cake. Let’s dig deeper. This is a long post, you’ve been warned. Test Environment I will cover full test results in the next article in this series. First, let’s look into the test setup, environment and ....
|
|
Congratulations, you have a lot of code!"Congratulations, you have a lot of code!" Remedying Android’s method limit - Part 1
-
developers.soundcloud.com
-
10 years ago
-
eng
At SoundCloud we have been building for the Android platform since 2010. Much has changed since then: the team has grown, the list of features has grown, and our audience has grown. Today, eight engineers are working full time on the official SoundCloud app, across various areas, with contributions pouring in from other parts of the organization. Due to the growing complexity and number of contributions, the app’s size has grown substantial....
|
|
h2Testing Approaches/h2 ol liemImmediately/em on the code that is being developed to help guide development. Generally using a emsubset/em of the test suite./li liemAfterwards/em on the complete codebase to ensure no regressions appear. This is always done using the emcomplete/em test suite./li /ol pIn an ideal environment, the complete test suite is run after every meaningful code change. This approach works for a small project, but it can....
|
|
Say we have a hungry caterpillar that wants to eat a leaf off of a tree. There are ten trees it can choose from. So, as far as we know, the probability of it choosing a specific tree is
|
|
Congratulations, you have a lot of code!"Congratulations, you have a lot of code!" Remedying Android’s method limit - Part 1
-
developers.soundcloud.com
-
10 years ago
-
eng
At SoundCloud we have been building for the Android platform since 2010. Much has changed since then: the team has grown, the list of features has grown, and our audience has grown. Today, eight engineers are working full time on the official SoundCloud app, across various areas, with contributions pouring in from other parts of the organization. Due to the growing complexity and number of contributions, the app’s size has grown substantial....
|
|
Recommended? yep There's a corner in Chinatown hosting some truly superb treats. If you get caught in the rush between Newport court and Newport place, you'd likely fail to notice some the awesome street food stands. Chilly squid I've walked past this place many times and never noticed it. They serve a handful of items, but the grilled chilly squid skewers caught my attention. They're grilled, brushed with chilly sauce and finished w....
|
|
I often like to think of software architecture as the art of structuring software. It's goal being to meet requirements while at the same…
|
|
I realized I didn’t actually include any implementation details when writing my sleepy post about stable locations and temperature the other night. I figured I’d elaborate on that now.
|
|
SICP in Clojure - Chapter 3 In one of the previous blog posts I have announced that I would like to start a new series of posts. It is a persistent journal from my journey through aforementioned book. I hope that you will enjoy it and find it useful - the main goal is to make this series a place where we can return in future, recall ideas and thoughts that accompanied reading process. Introduction In the previous blog post (which had s....
|
|
11 mistakes I’ve made during React Native / Redux app development . 9 things every React.js beginner should know (Hacker News) . 9 things every React.js beginner should know . A Complete Guide to Flexbox . A complete native navigation solution for React Native with optional redux support - nav bars, tabs, drawer, modals . A Material Design style React Native component library . A pull to refresh ListView for React Native . Adam Wolf'....
|
|
pThis week it#39;s been a hot topic of discussion about how Google has (almost) all of their code within a single, massive, 2-billion lines of code repository./p pThere was a quote by a href=https://twitter.com/searlsJustin Searls/a, which was most definitely him joking around, but it got me thinking about how we share code at theScore./p pThere#39;s the common misconception that you either have a quot;Monolithquot; or series of quot;Micros....
|
|
Getting to Know Fiddler: Part VIII: It isn't all rainbows and unicorns
-
joshuarogers.net
-
10 years ago
-
eng
Roughly a year ago I posted what I expected to be the last entry in my Fiddler series. Each of the seven entries had focused on various ways that Fiddler could help improve our development life cycle. They represented the highlight reel. All isn't perfect though, and where there is a highlight reel, there tends to be outtakes as well. So, here goes: here's a few ways you can use Fiddler to slow down your development process.
|
|
So you want a custom 404 page for your project page, or just any subdirectory? Well, according the GitHub help page, “you must use a custom domain.”
|
|
SnailLife snail stables have always lived in the ether - in some virtual universe with no physical location. Since I’m trying to make SnailLife based in reality, users’ snail stables should also be based in real locations. This is why now, upon registration, the user’s physical location is used as the location of their snail stable.
|
|
Build & Push a Docker Image using Visual Studio Online Build vNext
-
blog.gripdev.xyz
-
10 years ago
-
eng
So I’ve got a nice and simple NodeJs app and I want to have a CI build which builds my Docker image and pushes it to my docker hub ready for deployment. I’ll look at deployment in a future post, this time round we’ll focus on the build process. To get this setup we’ll need to go through two bits, basically VSO uses agents (machines used to execute builds) and a build, which defines some steps that output an artifact (in this case a docker i....
|
|
PostgreSQL has a great feature called Foreign Data Wrappers (FDW) that allows it to connect directly to outside systems. Although the setup can be a little complicated, once it’s available you can run queries with joins or subqueries against them, insert data, create views, etc. Heroku has dramatically simplified the process of using FDW with PostgreSQL and Redis thanks to Data Links. Let’s try it out.
|