Wyłowione z czytnika

Losing Andrew Carnegie

2010-03-07T10:43:00Z

Carnegie apparently said, "Take away my people, but leave my factories and soon grass will grow on the factory floors......Take away my factories, but leave my people and soon we will have a new and better factory."

Is there a typical large corporation working today that still believes this?

Most organizations now have it backwards. The factory, the infrastructure, the systems, the patents, the process, the manual... that's king. In fact, shareholders demand it.

It turns out that success is coming from the atypical organizations, the ones that can get back to embracing irreplaceable people, the linchpins, the ones that make a difference. Anything else can be replicated cheaper by someone else.

Gallery of Processor Cache Effects

2010-01-19T10:28:11Z

Most of my readers will understand that cache is a fast but small type of memory that stores recently accessed memory locations. This description is reasonably accurate, but the “boring” details of how processor caches work can help a lot when trying to understand program performance.

In this blog post, I will use code samples to illustrate various aspects of how caches work, and what is the impact on the performance of real-world programs.

The examples are in C#, but the language choice has little impact on the performance scores and the conclusions they lead to.

Example 1: Memory accesses and performance

How much faster do you expect Loop 2 to run, compared Loop 1?

int[] arr = new int[64 * 1024 * 1024];

// Loop 1
for (int i = 0; i < arr.Length; i++) arr[i] *= 3;

// Loop 2
for (int i = 0; i < arr.Length; i += 16) arr[i] *= 3;

The first loop multiplies every value in the array by 3, and the second loop multiplies only every 16-th. The second loop only does about 6% of the work of the first loop, but on modern machines, the two for-loops take about the same time: 80 and 78 ms respectively on my machine.

The reason why the loops take the same amount of time has to do with memory. The running time of these loops is dominated by the memory accesses to the array, not by the integer multiplications. And, as I’ll explain on Example 2, the hardware will perform the same main memory accesses for the two loops.

Example 2: Impact of cache lines

Let’s explore this example deeper. We will try other step values, not just 1 and 16:

for (int i = 0; i < arr.Length; i += K) arr[i] *= 3;

Here are the running times of this loop for different step values (K):

Notice that while step is in the range from 1 to 16, the running time of the for-loop hardly changes. But from 16 onwards, the running time is halved each time we double the step.

The reason behind this is that today’s CPUs do not access memory byte by byte. Instead, they fetch memory in chunks of (typically) 64 bytes, called cache lines. When you read a particular memory location, the entire cache line is fetched from the main memory into the cache. And, accessing other values from the same cache line is cheap!

Since 16 ints take up 64 bytes (one cache line), for-loops with a step between 1 and 16 have to touch the same number of cache lines: all of the cache lines in the array. But once the step is 32, we’ll only touch roughly every other cache line, and once it is 64, only every fourth.

Understanding of cache lines can be important for certain types of program optimizations. For example, alignment of data may determine whether an operation touches one or two cache lines. As we saw in the example above, this can easily mean that in the misaligned case, the operation will be twice slower.

Example 3: L1 and L2 cache sizes

Today’s computers come with two or three levels of caches, usually called L1, L2 and possibly L3. If you want to know the sizes of the different caches, you can use the CoreInfo SysInternals tool, or use the GetLogicalProcessorInfo Windows API call. Both methods will also tell you the cache line sizes, in addition to the cache sizes.

On my machine, CoreInfo reports that I have a 32kB L1 data cache, a 32kB L1 instruction cache, and a 4MB L2 data cache. The L1 caches are per-core, and the L2 caches are shared between pairs of cores:

Logical Processor to Cache Map:
*---  Data Cache          0, Level 1,   32 KB, Assoc   8, LineSize  64
*---  Instruction Cache   0, Level 1,   32 KB, Assoc   8, LineSize  64
-*--  Data Cache          1, Level 1,   32 KB, Assoc   8, LineSize  64
-*--  Instruction Cache   1, Level 1,   32 KB, Assoc   8, LineSize  64
**--  Unified Cache       0, Level 2,    4 MB, Assoc  16, LineSize  64
--*-  Data Cache          2, Level 1,   32 KB, Assoc   8, LineSize  64
--*-  Instruction Cache   2, Level 1,   32 KB, Assoc   8, LineSize  64
---*  Data Cache          3, Level 1,   32 KB, Assoc   8, LineSize  64
---*  Instruction Cache   3, Level 1,   32 KB, Assoc   8, LineSize  64
--**  Unified Cache       1, Level 2,    4 MB, Assoc  16, LineSize  64

Let’s verify these numbers by an experiment. To do that, we’ll step over an array incrementing every 16th integer – a cheap way to modify every cache line. When we reach the last value, we loop back to the beginning. We’ll experiment with different array sizes, and we should see drops in the performance at the array sizes where the array spills out of one cache level.

Here is the program:

int steps = 64 * 1024 * 1024; // Arbitrary number of steps
int lengthMod = arr.Length - 1;
for (int i = 0; i < steps; i++)
{
    arr[(i * 16) & lengthMod]++; // (x & lengthMod) is equal to (x % arr.Length)
}

And here are the timings:

You can see distinct drops after 32kB and 4MB – the sizes of L1 and L2 caches on my machine.

Example 4: Instruction-level parallelism

Now, let’s take a look at something different. Out of these two loops, which one would you expect to be faster?

int steps = 256 * 1024 * 1024;
int[] a = new int[2];

// Loop 1
for (int i=0; i<steps; i++) { a[0]++; a[0]++; }

// Loop 2
for (int i=0; i<steps; i++) { a[0]++; a[1]++; }

It turns out that the second loop is about twice faster than the first loop, at least on all of the machines I tested. Why? This has to do with the dependencies between operations in the two loop bodies.

In the body of the first loop, operations depend on each other as follows:

But in the second example, we only have these dependencies:

The modern processor has various parts that have a little bit of parallelism in them: it can access two memory locations in L1 at the same time, or perform two simple arithmetic operations. In the first loop, the processor cannot exploit this instruction-level parallelism, but in the second loop, it can.

[UPDATE]: Many people on reddit are asking about compiler optimizations, and whether { a[0]++; a[0]++; } would just get optimized to { a[0]+=2; }. In fact, the C# compiler and CLR JIT will not do this optimization – not when array accesses are involved. I built all of the tests in release mode (i.e. with optimizations), but I looked at the JIT-ted assembly to verify that optimizations aren’t skewing the results.

Example 5: Cache associativity

One key decision in cache design is whether each chunk of main memory can be stored in any cache slot, or in just some of them.

There are three possible approaches to mapping cache slots to memory chunks:

Direct mapped cache
Each memory chunk can only be stored only in one particular slot in the cache. One simple solution is to map the chunk with index chunk_index to cache slot (chunk_index % cache_slots). Two memory chunks that map to the same slot cannot be stored simultaneously in the cache.
N-way set associative cache
Each memory chunk can be stored in any one of N particular slots in the cache. As an example, in a 16-way cache, each memory chunk can be stored in 16 different cache slots. Commonly, chunks with indices with the same lowest order bits will all share 16 slots.
Fully associative cache
Each memory chunk can be stored in any slot in the cache. Effectively, the cache operates like a hash table.

Direct mapped caches can suffer from conflicts – when multiple values compete for the same slot in the cache, they keep evicting each other out, and the hit rate plummets. On the other hand, fully associative caches are complicated and costly to implement in the hardware. N-way set associative caches are the typical solution for processor caches, as they make a good trade off between implementation simplicity and good hit rate.

For example, the 4MB L2 cache on my machine is 16-way associative. All 64-byte memory chunks are partitioned into sets (based on the lowest order bits of the chunk index), and chunks in the same set compete for 16 slots in the L2 cache.

Since the L2 cache has 65,536 slots, and each set will need 16 slots in the cache, we will have 4,096 sets. So, the lowest 12 bits of the chunk index will determine which set the chunk belongs to (2¹² = 4,096). As a result, cache lines at addresses that differ by a multiple of 262,144 bytes (4096 * 64) will compete for the same slot in the cache. The cache on my machine can hold at most 16 such cache lines.

In order for the effects of cache associativity to become apparent, I need to repeatedly access more than 16 elements from the same set. I will demonstrate this using the following method:

public static long UpdateEveryKthByte(byte[] arr, int K)
{
    Stopwatch sw = Stopwatch.StartNew();
    const int rep = 1024*1024; // Number of iterations – arbitrary

    int p = 0;
    for (int i = 0; i < rep; i++)
    {
        arr[p]++;
        p += K;
        if (p >= arr.Length) p = 0;
    }

    sw.Stop();
    return sw.ElapsedMilliseconds;
}

This method increments every K-th value in the array. Once the it reaches the end of the array, it starts again from the beginning. After running sufficiently long (2^20 steps), the loop stops.

I ran UpdateEveryKthByte() with different array sizes (in 1MB increments) and different step sizes. Here is a plot of the results, with blue representing long running time, and white representing short:

The blue areas (long running times) are cases where the updated values could not be simultaneously held in the cache as we repeatedly iterated over them. The bright blue areas correspond to running times of ~80 ms, and the nearly white areas to ~10 ms.

Let’s explain the blue parts of the chart:

Why the vertical lines?The vertical lines show the step values that touch too many memory locations (>16) from the same set. For those steps, we cannot simultaneously hold all touched values in the 16-way associative cache on my machine.
Some bad step values are powers of two: 256 and 512. As an example, consider step 512 on an 8MB array. An 8MB cache line contains 32 values that are spaced by 262,144 bytes apart. All of those values will be updated by each pass of our loop, because 512 divides 262,144.

And since 32 > 16, those 32 values will keep competing for the same 16 slots in the cache.

Some values that are not powers of two are simply unfortunate, and will end up visiting disproportionately many values from the same set. Those step values will also show up as as blue lines.
Why do the vertical lines stop at 4MB array length?On arrays of 4MB or less, a 16-way associative cache is just as good as a fully associative one.
A 16-way associative cache can hold at most 16 cache lines that are a multiple of 262,144 bytes apart. There is no set of 17 or more cache lines all aligned on 262,144-byte boundaries within 4MB, because 16 * 262,144 = 4,194,304.
Why the blue triangle in upper left?In the triangle area, we cannot hold all necessary data in cache simultaneously … not due to the associativity, but simply because of the L2 cache size limit.
For example, consider the array length 16MB with step 128. We are repeatedly updating every 128th byte in the array, which means that we touch every other 64-byte memory chunk. To store every other cache line of a 16MB array, we’d need 8MB cache. But, my machine only has 4MB of cache.

Even if the 4MB cache on my machine was fully associative, it still wouldn’t be able to hold 8MB of data.
Why does the triangle fade out in the left?Notice that the gradient goes from 0 to 64 bytes – one cache line! As explained in examples 1 and 2, additional accesses to same cache line are nearly free. For example, when stepping by 16 bytes, it will take 4 steps to get to the next cache line. So, we get four memory accesses for the price of one.
Since the number of steps is the same for all cases, a cheaper step results in a shorter running time.

These patterns continue to hold as you extend the chart:

Cache associativity is interesting to understand and can certainly be demonstrated, but it tends to be less of a problem compared to the other issues discussed in this article. It is certainly not something that should be at the forefront of your mind as you write programs.

Example 6: False cache line sharing

On multi-core machines, caches encounter another problem – consistency. Different cores have fully or partly separate caches. On my machine, L1 caches are separate (as is common), and there are two pairs of processors, each pair sharing an L2 cache. While the details vary, a modern multi-core machine will have a multi-level cache hierarchy, where the faster and smaller caches belong to individual processors.

When one processor modifies a value in its cache, other processors cannot use the old value anymore. That memory location will be invalidated in all of the caches. Furthermore, since caches operate on the granularity of cache lines and not individual bytes, the entire cache line will be invalidated in all caches!

To demonstrate this issue, consider this example:

private static int[] s_counter = new int[1024];
private void UpdateCounter(int position)
{
    for (int j = 0; j < 100000000; j++)
    {
        s_counter[position] = s_counter[position] + 3;
    }
}

On my quad-core machine, if I call UpdateCounter with parameters 0,1,2,3 from four different threads, it will take 4.3 seconds until all threads are done.

On the other hand, if I call UpdateCounter with parameters 16,32,48,64 the operation will be done in 0.28 seconds!

Why? In the first case, all four values are very likely to end up on the same cache line. Each time a core increments the counter, it invalidates the cache line that holds all four counters. All other cores will suffer a cache miss the next time they access their own counters. This kind of thread behavior effectively disables caches, crippling the program’s performance.

Example 7: Hardware complexities

Even when you know the basics of how caches work, the hardware will still sometimes surprise you. Different processors differ in optimizations, heuristics, and subtle details of how they do things.

On some processors, L1 cache can process two accesses in parallel if they access cache lines from different banks, and serially if they belong to the same bank. Also, processors can surprise you with clever optimizations. For example, the false-sharing example that I’ve used on several machines in the past did not work well on my machine without tweaks – my home machine can optimize the execution in the simplest cases to reduce the cache invalidations.

Here is one odd example of “hardware weirdness”:

private static int A, B, C, D, E, F, G;
private static void Weirdness()
{
    for (int i = 0; i < 200000000; i++)
    {
        <something>
    }
}

When I substitute three different blocks for “<something>”, I get these timings:

<something>	Time
A++; B++; C++; D++;	719 ms
A++; C++; E++; G++;	448 ms
A++; C++;	518 ms

Incrementing fields A,B,C,D takes longer than incrementing fields A,C,E,G. And what’s even weirder, incrementing just A and C takes longer than increment A and C and E and G!

I don’t know for sure what is the reason behind these numbers, but I suspect it is related to memory banks. If someone can explain these numbers, I’d be very curious to hear about it.

The lesson of this example is that can be difficult to fully predict hardware performance. There is a lot that you can predict, but ultimately, it is very important to measure and verify your assumptions.

Conclusion

Hopefully all of this helps you understand how caches work, and apply that knowledge when tuning your programs.

Microsoft and Creative Destruction

2010-02-11T17:15:59Z

A recent NYT article by former Microsoft VP Dick Brass has caused quite the stir, but for the wrong reasons. Every follow up article I’ve read, including one from Microsoft, gets much of it wrong some key things wrong.

The premise: The core point of the Brass article is how the introduction of middle management and bureaucracy has killed innovation at Microsoft.

My counterargument: Microsoft has always been a conservative, platforms company. Visionary design and creative leaders think in terms of great products, which Microsoft has never been good at. Brass assumes the challenges that hampered Tablet PC were new and local, but they have always been there. Microsoft’s best, and most creative, work has come when a competitor forced one of the few Renaissance-VPs (VPs who were not over-promoted engineers but actually had a diversity of management skills) to take product design seriously.

My credentials: I worked at MSFT 1994 to 2003. I was on the IE 1.0 to IE 5.0 team among others (Windows, MSN, and MSTE/Best Practices, where I worked with many groups across the company). I wrote a bestselling book about Innovation and I’ve spoken and consulted with various groups at the company dozens of times since I left in 2003.

My take:

The primary problem at Microsoft regarding good design & innovation is the diffusion of creative authority. The problem is not the numbers of people at the company, or the layers of management, as many gripe about. Layers don’t help, but it’s not the problem. The real issue is the inability to grant creative authority to the few people worthy of it. Microsoft has always been a place that gives way too many people a say in matters of design, vision and user experience, and it shows in the pervasive mediocrity of the majority of its products. Films need directors. Orchestras need conductors. But if you divide things into 30 pieces and ask 30 people to play creative visionary, mediocrity ensues. The better products at Microsoft are the ones where VPs modify the distribution of authority to create clear creative authority.
Few VPs are qualified to be creative leaders, at Microsoft or elsewhere. And there is no creative lead role at Microsoft. There never has been. This is not new, it has always been true (at least since 1994 when I started). This is why when brilliant, genius type software designers come to the company, they are baffled by how little creative power they can earn, so they retreat to research or future thinking groups that have no skin in the game (e.g. Bill Buxton, Steve Capps, Ray Ozzie, Jim Gray (RIP), etc.). Microsoft is simply a hard place for to accumulate wide authority over design, which is required to make coherent visions, user experiences and innovations come true. Worse, it’s rare for leaders to acknowledge death by too many cooks since those who have never worked elsewhere, and have no conception of creative process, can’t imagine any other way. The culture has always been a heavily consensus/collaboration driven place for managers, which waters down ideas, and shifts what goes out the door heavily towards conservation.
Management at Microsoft is fat with inbred managers who are not worthy of their title, but this has always been true. If you are hired to manage version 5 of something, you inherit a host of decisions made with skills you do not have, yet get credit for anyway. If the team you inherit does good work, and you happen to be the manager, you receive credit, regardless of how little you did. Entire unprofitable, failed divisions, funded by the rest of the company, promote people out of corporate obligation, creating the existence of middle managers who have never actually successfully managed anything in the marketplace. For the 90s, this was MSN and Consumer products, which were perennial failures. The quality pool of people who managed in those divisions was below average and as the company aged more of these groups were born. Microsoft, like all companies, has suffered from the Peter principle, or worse, perhaps the Paul Principle (people who are lousy at even simple management skills but inherit mediocre projects they don’t understand, and simply manage not to get fired via their team’s noble but unheralded efforts, which hide their shortcomings). As a result, there are line level managers at Microsoft who are more competent than some middle or senior managers. But this has always been true, given the diversity of the company. It’s worse now because of the size.
Real layoffs would be a blessing. In 1999 when I left the Internet Explorer team (before the ill-fated IE 6.0 release), I looked around the company for other teams to work on. I couldn’t believe how many lost, misguided, sad, self-destructive teams I saw. This was in 1999! The company has more than tripled in size since then. Mini-Microsoft is so clearly on the mark about his core ambitions. I don’t wish unemployment on anyone, but I’d say a) the ratio of managers to programmers is insanely out of whack b) The number of projects and divisions that have never made profit and are market laggards is obscene. If the company were split apart, few groups are competent enough to survive a year. This defeats the “strategic value” these properties supposedly have, as dumping of buckets of money earned by Office and Windows profits into their bonfires of incompetence does not a strategy make. You need basic leadership competence, which all too many groups at Microsoft don’t have (and many never did).
Microsoft’s best and most inventive work has often been driven by competition. A visible and serious threat is the only situation where leadership, historically, was forced to be creatively aggressive, giving a chance for creatives to obtain enough power to do good work. Windows 95, Office 95, Internet Explorer 5.0, MS Natural Keyboard, XBOX 360 were all excellent products by most standards, and were made possible by strong competition. The question executives need to ask is why divisions like Mobile & MSN,or the entire Vietnam like 15 year history of imploding efforts of web search (there is a great book to be written by someone about this), have been disasters despite clear and strong competition – this is the analysis to post on every office door at the rest of the company.
It’s lazy arguing to assume an organization of 10,000 or 100,000 is uniform in any way. Groups at Microsoft have a different culture, and some have been wildly more successful than others (e.g. Office vs. MSN/Live/whatever it’s called this week) in part because their leaders have developed superior cultures that diverge widely from other groups. Windows 7 is an excellent product no matter how it stands in comparison to Apple’s work, and the turnaround from Windows Vista, which many heralded as the end of MSFT, was beyond noteworthy. If Windows 7 or XBOX 360 is made in the same company that makes all the products you hate, you have to realize the limits of painting broad strokes. This is where many critiques of Microsoft fall short, including the one by Brass. They assume uniformity, projecting a local set of experiences in part of the company as the model for the entire company.
If you talk only to people who quit and were disgruntled you can’t possibly have the whole story. I’ve never met Dick Brass, but I know the Tablet PC was a commercial failure. As smart as Dick is, its likely he never understood how IE beat Netscape (it was more than the monopoly stuff), or Office beat Lotus/WordPerfect etc. He also might not know the long history of Windows and Office rejecting most requests from most other teams as a matter of both basic sanity and arrogance. Specific to Tablet PC, it started as a Bill Gates pet project. Working with Bill, who Dick curiously never mentions, was no treat, and unlike Steve Jobs, his direct involvement in matters of design is likely not a godsend. Articles like this one reads too much into corporate policies, as many of them are old (e.g. the review process) and good managers have always had ways to work within these rules to reward good employees. I’d agree the processes could be improved, but all the good VPs find ways to bend rules into loopholes.
The greatest disease at Microsoft is lack of sharing lessons from failure, especially where innovation is concerned. Microsoft has made many big, visible bets. Many of them have failed, but that’s par for the course. The problem is these expensive lessons are swept under the rug, encouraging others in the company to repeat the same mistakes. Everyone loves to make fun of Microsoft Bob, but few can articulate why it failed. If you don’t understand why it failed, you don’t have any reason for laughing so hard, and you likely aren’t half as smart as you think you are. A case study on Vista, MSN Search, Microsoft Bob, The Tablet PC, etc. should be produced by an outside consultant, and stapled on the forehead of every manager at the company, once a day, until they read them all word for word. Then they’d take advantage of Microsoft’s so called experience and wisdom. Otherwise, they are being set up to make the same expensive mistakes again and again.
The idea of Innovation, and Innovation Systems, is a distraction. Success in the market is a better scorecard and the most reliable source of criticism. Innovation, as the word is used in these articles, is a matter of taste. You can be very inventive and still get your ass kicked. Or do a great job with mostly conventional ideas, and kick more interesting competitors off the field. Apple, if you study their choices, doesn’t pull things out of the sky (digital music players, cell phones, and tablet PCs were all established ideas). They enter games others are already playing and kick their ass. But innovation is the least useful lens. The best criticism of Microsoft’s management is how, or how not, they’ve done against their competitors in terms of customer satisfaction. If innovation matters as much as people seem to claim it does, it’s well reflected in either market success or customer satisfaction, so worry more about those solid measures, rather than the ethereal notion of who is innovative and who isn’t.

1024-bit RSA encryption cracked by carefully starving CPU of electricity

2010-03-09T07:47:00Z

Since 1977, RSA public-key encryption has protected privacy and verified authenticity when using computers, gadgets and web browsers around the globe, with only the most brutish of brute force efforts (and 1,500 years of processing time) felling its 768-bit variety earlier this year. Now, three eggheads (or Wolverines, as it were) at the University of Michigan claim they can break it simply by tweaking a device's power supply. By fluctuating the voltage to the CPU such that it generated a single hardware error per clock cycle, they found that they could cause the server to flip single bits of the private key at a time, allowing them to slowly piece together the password. With a small cluster of 81 Pentium 4 chips and 104 hours of processing time, they were able to successfully hack 1024-bit encryption in OpenSSL on a SPARC-based system, without damaging the computer, leaving a single trace or ending human life as we know it. That's why they're presenting a paper at the Design, Automation and Test conference this week in Europe, and that's why -- until RSA hopefully fixes the flaw -- you should keep a close eye on your server room's power supply.

1024-bit RSA encryption cracked by carefully starving CPU of electricity originally appeared on Engadget on Tue, 09 Mar 2010 02:47:00 EST. Please see our terms for use of feeds.

Permalink The Register, TechWorld | University of Michigan | Email this | Comments

Achievement Porn

2010-02-26T16:00:34Z

Hacker News pointed me to an article recently called “Addicted to Fake Achievement“. That article is about video games, but this one isn’t. The thesis of that article is that some games (it mentions Role Playing Games) don’t actually require skill, just time. You start with a weak character and perform some task that’s almost [...] Read More:

Meaningful Work

Czy czterogodzinny tydzień pracy w wersji dla inżynierów jest w ogóle możliwy?

2010-03-07T13:32:08Z

“4-hour workweek” to swego rodzaju fenomen ostatnich lat. Ten niepozorny tomik przypomniał wielu zapracowanym, skoncentrowanym na karierze ludziom, że przyjemność z życia można (a wręcz powinno się) czerpać tu i teraz. Dziś, kiedy człowiek jest wystarczająco młody, sprawny i pełen energii, a nie w bliżej nieokreślonym dniu “zarobienia miliona”, “przejścia na emeryturę” lub osiągnięcia innego hipotetycznego celu. Pierwsze wydanie w/w książki było bardziej praktyczne, techniczne i trochę surowe. Odniesiony sukces, tysiące fanów oraz ukuta gdzieś po drodze nazwa całego tego fenomenu: lifestyle design, skłoniły autora do przygotowania nowej poszerzonej edycji, której głównym mottem i myślą przewodnią stało się już oficjalnie “projektowanie stylu życia”. Chcąc przekonać czytelników, że każdy, bez względu na wykonywany zawód może “zarabiać na życie i jednocześnie zwiedzać świat” autor wrzucił cały szereg interesujących case studies. Niestety niektóre z nich, dla mnie jako osoby w temacie, trochę zgrzytają.

Case study wyssany z palca

Oto Pan Sherwood, z zawodu mechanical engineer, którego marzeniem jest podróżowanie po świecie i wykonywanie swojej pracy zdalnie. Realizując swój misternie przygotowany plan, Pan Sherwood przekonuje szefa, że kiedy nie ma go w firmie pracuje lepiej, wydajniej i tworzy dwa razy więcej projektów (bo nic go nie rozprasza) i że tak naprawdę to jego obecność w pracy nie jest konieczna. Ech, dawno nie czytałem takich bajek.

“Wielu najchętniej robi to, co nie wymaga pracy.”

– Władysław Grzeszczyk, “Parada paradoksów”

Może i byłaby to prawda, gdyby bohater tej historyjki zajmował się np. projektowaniem gwoździ. Niestety, mając za sobą pewne doświadczenie w pracy w zespole R&D, wiem jak taka praca naprawdę wygląda. Zdecydowanie inaczej niż w przedstawionym powyżej przykładzie. Przede wszystkim w żaden sposób nie da się jej skwantyfikować tak jak życzyłby sobie tego autor (w środę dwa projekty, a w czwartek trzy). Nad większością projektów działy R&D pracują całymi tygodniami lub miesiącami. Dni wypełnione są spotkaniami, grupowymi burzami mózgów, przygotowywaniem symulacji i prototypów, analizami matematycznymi, konsultacjami itp. Jedynie od czasu do czasu po prostu się czyta, studiuje materiał, zaszywa w jakimś kącie i po prostu myśli z ołówkiem w ręku – czyli to co w zasadzie można robić wszędzie. Historia Pana Sherwooda to wyssany z palca, zupełnie nierealistyczny przykład…

Poza tym działy badań i rozwoju to jedno. Czy potrafilibyście sobie wyobrazić inżyniera produkcji, który nadzoruje sytuację na linii z domu, a co dopiero z innego kraju?

Scott Ableman

Owszem są profesje i biznesy, które dają się wykonywać i zarządzać zewsząd – przede wszystkim te oparte na nowoczesnych technologiach sieciowych. Nie jest technicznie problemem pomieszkiwanie w Tajlandii i obsługiwanie serwisu internetowego w Europie. Niestety, kreatywna praca naukowca lub inżyniera (a więc ta na wysokim poziomie, prowadząca do powstawania nowych technologii i produktów, a nie poprawianie rysunków technicznych lub przygotowywanie instrukcji dla użytkowników) wymaga bycia na miejscu, generowania pomysłów, szerokiego dyskutowania, obrony lub krytyki różnych koncepcji, testowania modeli komputerowych oraz prototypów. Johnattan Ive i jego Design Team w Apple nigdy nie stworzyliby iPhone’a gdyby jego członkowie w międzyczasie zwiedzali sobie obce kraje, tudzież, jak Pan Sherwood, bawili na Oktoberfeście.

I dlatego właśnie nie da się żyć tak jak proponuje nam Tim Ferris, wykonując jednocześnie zawód bardzo mocno oddziałujący ze środowiskiem pracy, w którym intelektualne sprzężenie zwrotne w zespole nakręca postęp (pomijając już nawet inne istotne kwestie jak poufność materiałów oraz dostępność specjalizowanych narzędzi). Jeśli środowiskiem tym jest internet możesz mieszkać i pracować wszędzie. Zespół inżynierów i naukowców R&D pracuje razem, razem rozwiązuje problemy, podrzuca sobie nawzajem pomysły i wzajemnie się krytykuje. To właśnie ta sieć interakcji międzyludzkich pobudza kreatywność i motywuje do działania, a w konsekwencji jest źródłem przełomowych odkryć, rozwiązań i nowych technologii. To musi się odbywać twarzą w twarz. Przez telefon lub internet to nie to samo.

Mam wrażenie, że próbując rozszerzyć grupę docelową czytelników i dodając przykłady takie jak ten nieszczęsny Pan Sherwood, autor wkroczył na dość niepewny grunt. Rozumiem, że celem było doprecyzowanie i poszerzenie przekazu oraz zdecydowany skręt w stronę lifestyle design (wyleciał np. cały materiał o wirtualnych asystentach). Jednak przytaczanie wyimaginowanych case studies trąci marketingową manipulacją nie mającą wiele wspólnego z realizmem. I to jest właśnie powód, przez który zwykle przy lekturze blogów i książek promujących lifestyle design lepiej zachowywać wzmożoną czujność i duży sceptycyzm.

Alternatywa

Skończyłem niedawno lekturę “Linchpin” Setha Godina (kupiłem zachęcony fragmentami przytaczanymi przez Krzysztofa Wysockiego na jego blogu). Już od pierwszych stron rzuca się w oczy jak odmienne podejście do zagadnienia pracy (i życia w ogóle) ona reprezentuje. Seth Godin twierdzi, że automatyzacja procesów biznesowych i produkcyjnych zdewaluowała pracownika do “łatwo wymienialnego trybiku w maszynie”. W rezultacie prawdziwą wartość ma to co nie daje się zautomatyzować, to co jest wyjątkowe i unikatowe. Problem w tym, że tego typu praca wymaga kreatywności i niebanalnego postrzegania świata. Tylko wtedy jej rezultatem będzie coś, co autor określił po prostu dziełem sztuki.

Czy można można tworzyć sztukę (w rozumieniu Setha Godina) posługując się zautomatyzowanym procesem biznesowym, leżąc w tym samym czasie na plaży gdzieś tam? Jeśli coś jest zautomatyzowane, to jest dobrze opisane. Jeśli jest opisane – każdy może skorzystać z przepisu, uzyskując taki sam lub bardzo zbliżony efekt. Skoro może to robić każdy to jaka jest tak naprawdę wartość takiej pracy i jej owoców?

Seth Godin radzi – poszukuj wyjątkowości, bądź kreatywny. Twórz sztukę. Podnoś swoją wartość oraz wartość swojej pracy, tworząc nową lepszą jakość. Kiedy Twoja praca ma naprawdę wielką wartość, możesz wykonywać ją jak chcesz i kiedy chcesz – będziesz wolny.
Tim Ferris radzi – odrzuć reguły wg których funkcjonuje współczesny świat i nie daj się ograniczać swojej pracy. Zautomatyzuj co się da, aby zapewnić sobie stały pasywny dochód i mieć więcej czasu na robienie tego co chcesz i jak chcesz. Zdystansuj się do swojej pracy tak bardzo jak to możliwe – będziesz wolny.

Dwie drogi i dwa zupełnie różne podejścia do sprawy. Choć uważam, że książka i blog Tima Ferrisa to ciekawe pozycje, z których zdarzyło mi się nie jeden raz czerpać inspirację, to jednak zdecydowanie bardziej przemawia do mnie filozofia Setha Godina.

“Nie przepracowałem ani jednego dnia w swoim życiu. Wszystko co robiłem, to była przyjemność.”

– Thomas Alva Edison

Nie chcę tu jednoznacznie deklarować, że jedno podejście jest lepsze od drugiego. Są to po prostu dwa odmienne style funkcjonowania i każdy może wybrać to co bardziej mu odpowiada. W jednym przypadku “zrób tak, aby Twoją pasją była Twoja praca” a w drugim “zrób tak, aby Twoja praca nie przeszkadzała Ci w Twojej pasji”. “Linchpin” przemawia do mnie bardziej ponieważ bardziej realistycznie wpisuje się w środowisko pracy każdego z nas. Wystarczy uważnie rozejrzeć się dookoła, aby znaleźć wiele życiowych przykładów. Jak już pokazałem na przykładzie Pana Sherwooda, nie jest to takie oczywiste w przypadku “4-hour workweek”.

Zachęcam do wyrażenia własnej opinii w komentarzach.

Do poczytania

Setha Godin’s Blog
Tim Ferris’ Blog

Cold Remedy: 18 Real-World Lifestyle Design Case Studies (Now It’s Your Turn) (@4-hour workweek)

Prawie setka stron w nowej edycji “4-hour workweek” to materiały z blogu, więc jeśli chcecie dowiedzieć się więcej przed podjęciem decyzji o wydaniu pieniędzy , warto tam najpierw zajrzeć. Oto sześć z ośmiu tekstów, które znalazły się w książce (dwa brakujące stanowią cześć pozostałych):
The Art of Letting Bad Things Happen
Things I’ve Learned and Loved in 2008
How to Travel the World with 10 Pounds or Less (Plus: How to Negotiate Convertibles and Luxury Treehouses)
The Choice-Minimal Lifestyle: 6 Formulas for More Output and Less Overwhelm
The Margin Manifesto: 11 Tenets for Reaching (or Doubling) Profitability in 3 Months
The Holy Grail: How to Outsource the Inbox and Never Check Email Again

Artykuły o podobnej tematyce:

Porting to Python 3 — A Guide

2010-02-11T17:58:00Z

The latest Jinja 2 release came with basic support for Python 3. It was surprisingly painless to port the application over but it did require a substantial amount of tweaks and code changes in order to get it running. For everyone else out there who is interested in getting started, I decided to share my experiences:

Changing APIs

Before you start porting the library you have to decide how interfaces will behave in Python 3. The biggest issue here is obviously unicode, but there are others as well. I would say there are four kinds of libraries you might encounter regarding string behavior in Python 2: There are the libraries that only accept unicode and only output unicode, there are those that only accept byte-strings and output byte-strings but operate on textual data, there are the libraries that operate on either or and what has been fed into it, comes out of it and there are libraries that operate either on unicode or byte-strings and also accept the other type as long as it's a subset of the default encoding (ASCII).

First you have to find out what your library does, what it is supposed to do, and how you want to deal with that in Python 3. Because byte-strings no longer exist in Python 3 and were replaced by a bytes object that works similar, but has an incompatible API it is very unlikely that your code will be able to support both in the future (or that it is something you would desire).

Byte-Based Libraries

This is might the most tricky one if you are aiming for Python 2.5 support or lower and you are operating on bytes directly. The issue is that the way you operate on bytes changed fundamentally from Python 2.x to 3.x and 2to3 is not really able to pick it up. Worse, it will try convert all your bytestring literals to unicode! The official support is as far as I know, to explicitly prefix the byte strings in the 2.x code with a leading b to indicate bytes. Unfortunately that means no support for 2.x. I am not completely sure what to do in that situation, but at least I found a way to trick python to operate on bytes: if you have code like this:

magic = 'M23\x01'

And you want to ensure it does not end up being a str in 3.x, add a dummy encode:

magic = u'M23\x01'.encode('iso-8859-1')

The only downside is that the encode happens at runtime, so it will slow down execution a bit.

Text Based Libraries

The second kind of library is a library that operates on text. In 2.x there were multiple ways to implement such libraries and it basically came down to what data type was used internally and what was accepted for input and output. There are the libraries that operate exclusively either on bytestrings or unicode. These are the ones that are the easiest to port, because 2to3 was written with nearly that in mind. If your library was only accepting bytestrings in 2.x it will (after a 2to3 run) only be accepting a Python 3 str type which is unicode based. This works well as long as you do not intend to use some kind of IO in your library. Once you start doing that, you will need to make sure you can somehow specify the encoding to be used when opening files. In that case, make sure you open the file in byte mode (not in text mode!) and do the decoding/encoding yourself. This is the only way your IO code will work the same in both 2.x and 3.x. But more on IO later.

What 2to3 does out of the box is converting calls from unicode to str automatically. Unfortunately it does not change the special __unicode__ method to __str__. You can easily do that in a custom fixer though, so it should be easy to accomplish. If your library however supports both __str__ and __unicode__ you are in a more tricky situation here. Let me show you an example of the kind of classes I deal with in Jinja 2 for example:

class MyObject(object):

    def __init__(self):
        self.value = u'some value'

    def __str__(self):
        return unicode(self).encode('utf-8')

    def __unicode__(self):
        return self.value

The big problem here is that 2to3 will convert it to this:

class MyObject(object):

    def __init__(self):
        self.value = 'some value'

    def __str__(self):
        return str(self).encode('utf-8')

    def __unicode__(self):
        return self.value

If you call str() on your instance now, it will die with a runtime error because it recurses infinitely. Even if it would not recurse, it would try to return a bytes object from the __str__ method because of the encode call. My plan was to write a custom fixer that, if it detects a __str__ that just calls into __unicode__ and encodes, will drop the __str__ method and rename __unicode__ to __str__. Unfortunately the tree you are dealing with in 2to3 does not appear to be designed to removing code so what I do instead of removing the __str__ is just renaming the __unicode__ to __str__ and let Python override the dummy __str__ with the correct one. The fixer I use for that, looks like this:

from lib2to3 import fixer_base
from lib2to3.fixer_util import Name

class FixRenameUnicode(fixer_base.BaseFix):
    PATTERN = r"funcdef< 'def' name='__unicode__' parameters< '(' NAME ')' > any+ >"

    def transform(self, node, results):
        name = results['name']
        name.replace(Name('__str__', prefix=name.prefix))

After conversion with this fixer in place, the class from above will then look like this:

class MyObject(object):

    def __init__(self):
        self.value = 'some value'

    def __str__(self):
        return str(self).encode('utf-8')

    def __str__(self):
        return self.value

But where to put those fixers? Edit 2to3 directly? And do I have to provide two source packages for 2.x and 3.x? This is where distribute comes in.

2to3 through distribute

Distutils itself already has the possibility to run 2to3 for you, but what it cannot do is adding custom fixers without a lot of custom code. distribute on the other hand not gives you built in 2to3 support as a single keyword argument to setup() but can also pass custom fixers to 2to3 which is very helpful. Because these new keyword arguments however would warn if the setup script was executed with setuptools instead of distribute, you should only pass them to the setup function if invoked from Python 3. The setup script then looks like this:

import sys

from setuptools import setup

# if we are running on python 3, enable 2to3 and
# let it use the custom fixers from the custom_fixers
# package.
extra = {}
if sys.version_info >= (3, 0):
    extra.update(
        use_2to3=True,
        use_2to3_fixers=['custom_fixers']
    )


setup(
    name='Your Library',
    version='1.0',
    classifiers=[
        # make sure to use :: Python *and* :: Python :: 3 so
        # that pypi can list the package on the python 3 page
        'Programming Language :: Python',
        'Programming Language :: Python :: 3'
    ],
    packages=['yourlibrary'],
    # make sure to add custom_fixers to the MANIFEST.in
    include_package_data=True,
    **extra
)

Now all you have to do is to put the custom 2to3 fixers (written in Python 3!) into the custom_fixers package next to your real library and they will be added automatically. For examples of fixers, look into the lib2to3/fixes package or your Python 3 installation. If you run python3 setup.py build it will run 2to3 on your files and put the output into the build folder for you to test.

Input/Output

So in Python 3 there is a completely new input/output system. It is very Java-ish and is able to deal with unicode. The downside is that you either don't have it in 2.x or the implementation is too slow, so what you want to do is to create yourself an abstraction layer.

If your library was unicode based in older Python versions you probably just did file.read().decode(encoding) or something similar. This still works on 3.x and I strongly recommend doing that, but be sure to open the file in binary mode, otherwise on Python 3 the decode will attempt to decode an already decoded unicode string, which does not make any sense. If you need normalized newlines (windows newlines converted to '\n') you would have to post-process the string by hand, but must applications and libraries are able to deal with any kind of newline anyways.

You could also just create a IO helper module that calls the builtin open on 3.x and codecs.open on 2.x. Unfortunately codecs.open has a worse performance than the built in open on 2.x, so you might want to check how you are dealing with files, if a high performance is necessary and so forth. Most of the time, opening the file in binary mode is what you want to do.

If you library was byte based in 2.x and you opened files in the library, instead of just working on open file objects, you will have to change your API slightly in order to take the charset and error mode into account. If you previously had a function like this:

def read_file_contents(filename):
    with open(filename) as f:
        return f.read()

You will have to change it to something like this now:

def read_file_contents(filename, charset='utf-8', errors='strict'):
    with open(filename, 'rb') as f:
        return f.read().decode(charset, errors)

And then ensure that you give the user to provide these arguments to the function. This means that whatever calls this, would also have to accept this arguments and so forth. Not everyone is using utf-8, there might be legacy files in iso-8859-1 a user might still want to be able to open. With a proper error handling system, it might even be possible to fall back to another encoding if it does not decode as utf-8 properly.

Last but not least, 3.x StringIO is a "string IO", not something that accepts binary data. If you have a lot of unittests that are dealing with binary data in such objects, you will have to use the io.BytesIO instead. If it does not exist, you are running 2.x, and you can safely fall back to cStringIO.StringIO.

Unit-Testing

Now the biggest problem I had with switching to 3.x: The unittests. First of all: do not use doctest. There is a doctest converter in 2to3, but it does not give you much. Error messages changed, reprs changed which it cannot properly pick up, nested tracebacks cause a lot of grief and they are hard to debug. I was playing with the idea to write a tool that automatically converts doctests to unittests, but I was too lazy and converted the few I had in my code, to unittests by hand. Furthermore, the few doctests left (used as code examples in the documentation) are only tested if the testsuite is invoked from Python 2.x

Nosetest has 3.x support in a separate branch, py.test comes with 3.x for a while now and the builtin unittest does the trick as well. I personally converted all my Jinja 2 tests to unittest lately. If you are using unittest you can point distribute to your test suite function and it will run the test for you if you write python setup.py test. This even runs 2to3 for you if you execute it with Python 3. So very helpful.

Hope that helps you porting your libraries to Python 3. Would love to hear about your experiences, because even if Python 3 did not work out as some of us hoped, it is very important that we continue to port libraries over to 3.x.

The Strategy Trap: Why focusing too much on strategy could be killing your ability to execute.

2010-03-01T19:46:11Z

“They were worried that I would get bogged down in wanting to do things, not just create strategy.”

- David Polinchock / @lbbinc

One of the topics covered during the #LikeMinds Summit this past weekend was precisely this: The chasm between strategy and execution, especially as businesses struggle to understand how to leverage, integrate and operationalize Social Communications (what you do with social media platforms) in the coming 6-24 months.

Unfortunately, because the C-suite tends to look to itself when it comes to “strategic masterminding,” the focus too often shifts from execution at the customer level (the most important thing a business should be focusing on on) to… being the guy who came up with the game-changing strategy that will secure more funding and increase influence within the organization.

When this happens, strategy becomes a product, and that’s bad. Strategy isn’t a product. Strategy exists mostly in support of execution.

Any idiot with a powerpoint deck can deliver a “Social” strategy:

“We’ll create a facebook fan page, a twitter account, a LinkedIn Group, a YouTube channel, a blog, and a Flickr account too! We’ll convert all of our customers who participate in social networking into fans and followers, and we’ll engage them with content at regular intervals throughout the day. We’ll embed hyperlinks into our tweets and facebook updates so we can pull them to our website to increase traffic there. We’ll get lots of extra clicks. We’ll gain mindshare by being there with them on their favorite social platforms. When they talk to us, we’ll respond. We’ll monitor sentiment and mentions. Our social media strategy will be a success.”

Um… yeah, except… no.

Sometimes, companies focus so much on developing and implementing strategies that they forget to focus on what’s important: Focusing on the customers. That’s priority numero uno. As a consumer, I don’t give two shakes what a company’s latest strategy is. I really don’t care. You want to gain 4% market share in the next quarter? You want to dominate the tablet PC market? Okay. Great. What’s that to me? All I want is for you to improve my life. How are you planning on doing that? How does your strategy actually make anything happen on the ground? Have you thought about what happens when your theories actually touch the real world?

The gap between high level strategy and ground-level execution can usually be summed up this way: Do you understand the tactics and ground level dynamics enough to ensure that your strategy will turn into something more than just an inspiring powerpoint presentation? Yes = small or no gap. No = huge gap.

On the ground, in the real world, what does your grand strategy do to make me want to spend more time recommending you to my friends? Spend more time wishing I could fill my garage with more of your stuff? What’s your strategy to make my experiences with your brand outclass and outshine my experiences with every other company? What’s your strategy to be awesome?

Don’t just look at strategy from the top down and the inside out. Also look at it from the outside in. How does it play in terms of influencing customer perceptions and behavior? How does it differentiate you or increase preference?

Let me illustrate the difference between tactically-agnostic strategy and tactically-savvy strategy:

What could you do TODAY that would change the way customers feel about you?

a) Give them a 10% off rebate that may take 30-60 business days to process. (We’ll worry about eroding margins and loyalty later.)

b) Knock their socks off with incredible customer service. (Smiles are free and being helpful makes customers come back.)

- or another choice -

a) Try to nickle-and-dime a guest with a $10 bottle of water in their hotel room (hey, going after that incremental revenue looks genius on Excel. Let’s charge extra for everything! We’ll make billions off premium pillow mints.)

b) Slap a note on the bottle that says “It’s water. Of course it’s free.” (The repeat business, loyalty and recommendations are worth more than the odd begrudged transaction.)

Which hotel chain is more likely to get repeat business and earn recommendations?

Which of these options do you think a disconnected top-down strategy might have generated? a) or b)?

Strangely, few companies have an “awesomeness” strategy. They have growth strategies, sales strategies, reach strategies, campaign strategies, pull strategies… all of which include a lot of content creation, content distribution and push/pull schemes created and directed from the top echelons. Great stuff, don’t get me wrong. But also lots of wasted energy working its way down to customers through less than fluid “channels.” Lots of energy wasted encountering friction and resistance on their way to the customers. Encountering snags and problems. That’s the execution gap. That’s the part of implementation that too much emphasis on strategy, coupled with an operational chasm between the “strategists” and the “doers” creates.

So, your company isn’t short on strategy. You have dozens if not hundreds of powerpoint presentations to prove it. The quarterly deluge of strategic plans and “bullet points” and budget proposals to prove it. Social Media-related or otherwise. How’s that been working out for you?

Social Media – as it relates to Brand Management, PR, Marketing, Business Development, Community Management, recruiting, internal collaboration, product innovation and crisis management – isn’t about developing the winning strategy. There’s no “win” in developing or delivering a strategy. Any strategy. Ever. Anywhere. I mean, yes, you’re smart. We get it. Thank you. That’s wonderful. But now what?

The reality here, the nugget, is this: The emphasis on top-down strategy is completely wrong for Social Media and Social Communications. The way to truly make Social Media and Social Communications WORK for business requires a focus on enablement, not strategy. Strategies don’t generate revenue. Strategies don’t win market share. Strategies don’t make customers loyal. Strategies are bullets on a slide, ink on paper, words across a conference table. They’re essentially worthless until you can use them to move a needle.

The disproportionate investment in strategy vs. implementation and execution is at the heart of why “Social” works for so few companies right now.

1. What are we trying to improve? (What should we be trying to improve?) <– Start with customer experience. Always.

2. What will it take to make that happen?

3. Does Social fit in?

4. If so, how?

5. What can Social help us improve?

6. What will it take to make that happen?

That’s it. Those those 6 questions. Start there. Stop talking about it. Move towards something your customers will be able to grasp, enjoy, value and convey.

Next time someone wants to sell you on a strategy, tell them to come back when they can show you exactly how they plan to implement it. Always make the strategist responsible for the execution. You’ll save yourself a lot of headaches, and things will get done a lot faster.

Cheers,

Olivier

PS: The upcoming Red Chair double-workshop in Portland, Oregon (PDX) on March 11 and March 12 focuses on precisely that: How to actually put all of this into action. How to make it work. One session is designed for enterprise space management and executives, and the other for account management and Social Media for small business. It would be lovely of you to help spread the word, even if you can’t attend this time around. For registration and information, click here.

Filed under: Social Communications, social media, strategy Tagged: #RedChairPdx, beth harte, brandbuilder, chris brogan, david polinchock, execution, leadership, management, marketing, mike wagner, olivier blanchard, Red Chair, Social Communications, social media, strategy, tactics

The 22 minute meeting

2010-03-05T07:51:42Z

No one likes meetings and for good reason. In most meetings, most of the time, most people think most of what goes on is a waste of time.

So what if you took out all of the stupid, wasteful stuff and left only the useful parts?

Enter the 22 minute meeting. This is an idea from Nicole Steinbok, and she presented the idea at Seattle Ignite 9.� When I saw her present this concept at Microsoft a few months ago, she gave one of the best short talks I’ve ever seen.

Here’s the poster from her talk:

I couldn’t find a write up of the core points, so here’s my take on her ideas from what I remember from her talk. All credit should go her way:

Schedule a 22 minute meeting - Who decided meetings should be 30 or 60 minutes?� What data is this based on? None. 30 and 60 minute meetings leave no time to get between meetings, and assumes, on average, people need an hour to sort things out. Certainly not all meetings can be run in 22 minutes, but many can, so we’d all be better off if the default time were small, not large.
Have a goal based agenda – Having an agenda at all would be a plus in most meetings. Writing it on the whiteboard, earns double pluses, since then everyone has a constant reminder of what the meeting is supposed to achieve.
Send required readings 3 days beforehand – The burden is on the organizer to make this small enough that people actually do it. Never ever allow a meeting to be “lets all read the documents together and penalize anyone diligent enough to do their homework”. (note: I think 24 hours is plenty).
Start on time – How often does this happen? Almost never. Part of the problem is Outlook and all schedule programs don’t have space between meetings. By 2pm there is a day’s worth of meeting time debt. 22 minutes ensures plenty of travel/buffer time between meetings.
Stand up – Reminds everyone the goal isn’t to elaborate or be supplemental (See Scrum standing meetings). Make your point, make your requests, or keep quiet. If there is a disagreement, say so, but handle resolving it outside of the meeting.
No laptops, but presenters and note takes. If you’re promised 22 minutes, and it’s all good stuff, you don’t need a secondary thing to be doing while you pretend to be listening. One person taking notes, and one person presenting if necessary.
No phones, no exceptions – see above.
Focus! Note off topic comments.� If you have an agenda, someone has to police it and this burden is on whoever called the meeting. Tangents are ok, provided they are short. The meeting organizer has to table tangents and arguments that go too far from the agenda.
Send notes ASAP – With 22 minutes, there should be time, post meeting, for the organizer to send out notes and action items before the next meeting begins.

What do you think?

If you like the idea, help it spread. Nicole started a facebook group and a poster you can download (PDF).� Pass it on.

When/If they post her ignite talk online, I’ll post it here.

Designing User Interfaces For Business Web Applications

2010-02-25T13:56:00Z

Business Web application design is too often neglected. I see a lot of applications that don’t meet the needs of either businesses or users and thus contribute to a loss of profit and poor user experience. It even happens that designers are not involved in the process of creating applications at all, putting all of the responsibility on the shoulders of developers.

This is a tough task for developers, who may have plenty of back-end and front-end development experience but limited knowledge of design. This results in unsatisfied customers, frustrated users and failed projects.

So, we will cover the basics of user interface design for business Web applications. While one could apply many approaches, techniques and principles to UI design in general, our focus here will be on business Web applications.

[By the way: The network tab (on the top of the page) is updated several times a day. It features manually selected articles from the best web design blogs!]

Websites vs. Web Applications

Confusing Web applications and websites is easy, as is confusing user interface design and website design. But they are different both in essence and in so many other ways, which we’ll explore in this article.

A website is a collection of pages consisting mostly of static content, images and video, with limited interactive functionality (i.e. except for the contact form and search functionality). The primary role of a website is to inform. Some websites use content management systems to render dynamic content, but their nature is still informational.

CampaignMonitor is powerful email marketing software, while Jeff Sarimento’s website is intended to inform readers about his life and work.

Web applications, on the other hand, are dynamic, interactive systems that help businesses perform business critical tasks and that increase and measure their productivity. Thus, the primary role of a Web application is to perform a function that serves the user’s tasks and according to defined business rules.

Web applications require a higher level of involvement and knowledge of the system on the part of the user. They don’t just stumble upon the application, do their work and bounce off. They use it as a tool to perform critical business tasks in their daily work. In the end, they cannot easily discontinue using the application and switch to another if they don’t like how it’s working, as is the case with websites.

Different Types of Web Applications

Business applications range in type from invoicing for freelancers to content management systems to document management systems to banking and financial systems.

We can distinguish between open and closed applications. Open systems are online applications that are easily accessible to anyone who opens an account. Users can access such applications via the Web and can open an account for free or by paying a fee. Closed systems (or line-of-business applications) are usually not accessible outside the company that uses it, and they can be considered “offline” applications (though many systems expose their functionality to business partners via either services or specialized interfaces). Such systems usually run on the company’s local network and are available only to employees.

I don’t know who coined it, but one term I like especially is weblication, which describes what a Web application is in general. This doesn’t mean, though that a Web application is a half-website half-application hybrid. It is far more complex that that.

First, Know Your Users

You’ve probably heard this a thousand times, and for good reason. A successful user interface focuses on users and their tasks. This is key, and too many developers have failed to create a good user experience. As Steve Krug said, “Developers like complexity; they enjoy discovering how something works.”

When identifying your users, keep in mind that clients are not users, and you are not a user. Although a client’s management team will usually be interested in the project and try to influence decisions, remember that they won’t be sitting in front of the computer several hours a day (unless the application is specifically for them).

How to Identify Users?

Identifying users can be done using several techniques, such as user interviews, business stakeholder interviews and and the “shadowing” method of observation. Interviews can give you answers to questions about the users’ knowledge of the system and computers in general, while shadowing can yield more detailed information about how users perform tasks and what errors they make. The method is called shadowing because the observer is like a shadow, watching and noting the steps a user takes.

If you don’t have access to real users—either because you don’t have permission or are designing for open application—you can use personas, a tool to help identify users. Personas are a representation of real users, including their habits, goals and motivation. Because certain information about users is often identified through business analysis, you can make use of it to create personas. If you are not familiar with the tool, a comic by Brad Colbow will help.

Task analysis helps identify what tasks users perform in their jobs, how they do them, how long they take and what errors they make. Sometimes clients will be using an old version of the application that you are designing to replace. Make use of that old system and watch how users use it. Understanding their tasks and challenges will be easier that way.

Regardless of who your users are, one thing is certain: in most cases, you will have to consider both novices and experts. Novice users should be enabled to learn as fast as possible, while expert users should be enabled to perform their tasks extremely efficiently. This may mean creating separate interfaces. But in many cases you will be able to accommodate both types of users in the same interface through various techniques, such as progressive disclosure.

Such research is usually done by business analysts. But if no one else is responsible for it, you should do it. Once you have the necessary information, you can begin with design.

Design Process

You can follow one of any number of processes in designing the user interface. You might already have one. However, I would suggest that you consider the Agile approach. Why, you ask? Well, because for users (and clients), the user interface is the product. The bottom line is that they don’t care about your sketches or about fantastic back ends or powerful servers. All they want to see is the user interface.

So, how does Agile help? It helps through its key principle: the iterative approach. Each iteration consists of all of the phases defined by your process. This means that at the end of the first iteration, you will have a product that can be tested, a prototype.

Sketching

Sketching is a powerful way to explore ideas. The goal is to arrive at the solution by sketching out different concepts. Most sketches will be thrown out, but that is okay. As Bill Buxton says in his “Sketching User Experience” book, sketches are fast to create and easy to dispose of, which is why they are so powerful.

Are sketches the same as wireframes? Well, the differences can be blurry, but I would say no. Wireframes don’t capture rough ideas but rather develop them. Read a fantastic discussion on IxDA: Sketching Before the Wireframes.

Once you get the “right” sketches, or at least the ones that you think are right, you can create more detailed wireframes or go straight to creating interactive prototypes.

Sketch by Jason Robb.

Interesting reading on sketching and wireframing:

Prototyping

The next step in the process is to create prototypes that will simulate the real application. A prototype can contain one or more features (or all of them), but it actually does nothing. It merely simulates the behavior of a real application, and users will feel that they are actually doing something. Prototypes may contain some functionality if needed (such as complex calculations).

Because the nature of a prototype done in HTML is temporary—its purpose, after all, is to test ideas—don’t bother with the code; just make it work with minimal bugs. You will throw it away anyway. You can also use specialized prototyping software such as Axure. Some people even prototype in PowerPoint.

An Axure interactive prototype for an e-commerce website, by e-maujean.

Further reading and tools for prototyping:

5 Useful Online Tools for Web Design Planning and Prototyping
A Practitioner’s Guide to Prototyping: A book from Rosenfeld Media
16 Design Tools for Prototyping and Wireframing

Testing

Prototypes are useless unless you test them. This is not rocket science. People like Jakob Nielsen and Steve Krug support so-called “discount usability testing,” which is cheap and fast and yields valuable insight into your design decisions. You will use this information as the basis of another iteration of sketching, prototyping and testing. Do this at least until major issues have been fixed. We all know that software projects are tight on time and budget, so to be more efficient, test early and test often.

One of the best resources for discount usability testing is a new book by Steve Krug, “Rocket Surgery Made Easy.” Pick up a copy and read it.

Snapshot of usability testing for Delicious, by (nz)dave.

Further reading:

Design Principles

There are many design principles, but there doesn’t seem to be a general consensus on definitive ones. So, we’ll go through design principles more informally, leaving out strict definitions.

No One Likes Surprises

Probably the key factors in a good UI are consistency and familiarity. A user interface should be consistent across all parts of the application, from navigation to color to terminology. This is known as internal consistency. But a user interface should also be consistent within its context, such as the operating system or other applications in its group or family. A typical example is the applications in the Microsoft Office family. This is called external consistency.

A good approach to consistency is to define user interface guidelines for each project or for a group of projects. These should guide the decisions you make for all of the details. This will not only maintain consistency but also serve as documentation to help team members better understand your decisions.

Although a simple example, SprinklePenny achieves consistency and familiarity across the application.

Consistent user interfaces have a shorter learning curve, because users will recognize parts of the system and be able to fall back on prior experience. But familiarity is sometimes confused with consistency. Familiar user interfaces draw on concepts from the users’ previous experiences and use appropriate metaphors. Folders, for example, are a well-known metaphor for file organization, and they have replaced “directories,” which were used previously in command-line operating systems. In short, speak the language of your users.

A common belief among business owners is that a great user interface should look like a Microsoft Office product, especially Outlook. I won’t go into explaining how pointless this is. Rather, I will offer alternative advice: defend the user-centric approach, and explain why creating an application for employees, clients and partners (i.e. their users) is so important.

All the same, most businesses are unique, as are their work processes. For example, two businesses from the same branch could have significantly different processes, forcing you to go beyond what is familiar and start to innovate. This part of the design process can be very interesting, although you have to be careful in how far you go with innovation.

Further reading:

Users Should Be Able to Be Efficient

Without a doubt, users should be able to be efficient when using business applications. This is what they are paid for, and this is what managers expects from the application. User interfaces should allow users to be efficient and should focus them on completing tasks in the easiest and fastest way. But this is not always the case. There is an opinion, or at least practice, among developers that says the user interface should be as complex as the back end system. No matter how ridiculous this sounds, the problem is real and might give you a headache. This is one reason why good communication and collaboration between developers is a must.

Users are efficient when they focus on a particular task. As mentioned, task analysis can help you identify tasks and determine how users perform them. If tasks are long, accelerate them by breaking them up into smaller units. You can also increase efficiency by providing keyboard support and shortcuts. Think how inefficient it is for a user to have to switch back and forth between mouse and keyboard. In some cases, you will be designing for users who are accustomed to working on command-line operating systems and the applications made for them. They will be keen to have keyboard support. One suggestion: when defining keyboard shortcuts, keep them consistent with those of common applications. For example, Ctrl + S should always be save, and so on.

Google Docs Spreadsheet enables users to be efficient by providing keyboard shortcuts and context menus, as well as by taking advantage of users’ familiarity with common desktop applications.

Efficiency can also be enhanced through personalization. Users who can personalize an environment will learn it faster and, more importantly, will be more confident using it. Personalization can be done in many ways: choosing widgets for the dashboard; defining shortcut options and favorites; changing the order of elements; etc.

Pay attention to accessibility. Although many assume that accessibility doesn’t matter in Web applications, it certainly does. Treat the application as if it were a public website.

A Web application also has to be efficient in the speed with which it processes information. So, consider heavy interactions that result from partial renderings and AJAX requests.

Help!

An interface should provide meaningful feedback that describes the state of the system to users. If an error occurs, users should be notified and informed of ways to recover. If an operation is in progress, users should be notified about the progress.

We can go even further and declare that user interfaces should prevent users from making errors. This principle, called forgiveness, can be followed with confirmation dialogs, undo options, forgiving formats and more. Forgiveness makes it safe to explore the interface, decreases the learning curve and increases overall satisfaction.

Because of the complexity of business Web applications, you would also need to provide a comprehensive help system. This can be done with inline help, a support database, a knowledge base and guided tours (which mix video, images and text).

Further reading:

Can’t Get No Satisfaction

Satisfaction is a subjective term that refers to how pleasant an interface is to use. Every design principle we have described here affects satisfaction. And a few more principles are worth mentioning here.

Simplicity is a basic principle of UI design. The simpler a user interface, the easier it is to use. But keeping user interfaces for business applications simple is a challenge because the apps often have a lot of functionality. The key is to balance functionality and simplicity. Restraint is one of the most efficient ways to achieve this balance: i.e. finding the simplest way to solve a problem.

BuildWith.me has a simple and effective user interface, without sacrificing aesthetics.

Aesthetics, though subjective and somewhat arbitrary, play an important role in overall satisfaction. Users respond positively to pleasing user interfaces, sometimes even overlooking missing functionality. But you’re not creating a work of art. One of the best articles that explains aesthetics is In Defense of Eye Candy.

In the end, users will be spending a lot of time in front of a business application, and no matter how usable, consistent or forgiving the interface is, satisfaction will be critical in determining how good the user interface is.

Further reading:

Other resources related to UI design:

Essential Components Of Web Applications

Every Web application is unique, but many of them contain common features. Although the implementation of any one of these features will vary, let’s look at the basic concept of the three most common ones.

Web Forms

Forms in general are important to Web applications. But as Luke Wroblewski says in his Web Form Design book, “No one likes filing in forms.” That includes sign-up forms, including business applications with dozens of fields.

Minimize the frustration of filling in forms. Provide inline validation and good feedback. Use defaults when possible. Don’t forget about novice users. Use wizards to help them complete tasks faster, or use progressive disclosure to hide advanced (or infrequently used) features.

Master-Detail Views

This is the technique of showing data in two separate but related views. One view shows a list of items, while the other shows details of the selected item. Master-detail views can be implemented across multiple pages or on individual ones.

Dashboards

Many Web applications have dashboards. A dashboard is a view of the most important information needed to take action and make decisions. It is confined to a single page and is usually the starting point of an application. Dashboards are important because they enable users to access information and take action without having to dig through the application.

Xero shows a user’s most important financial information (e.g. bank accounts and credit cards) in its dashboard, making it easy for users to quickly see the status of their financial data.

Heavy Use of Tables

Because Web applications typically deal with large quantities of data that are easily accessible and sortable, tables are unavoidable. But this is not a bad thing. In fact, tables were made for this purpose. Don’t confuse this with table-less layouts. Effective tables are easily readable. So, in most cases you will need a meaningful header, an optimal number of columns, pagination, alternating row colors, proper column alignment, sorting and filtering capabilities and much more.

Tables can also be interactive, meaning they can generate additional info and even modify the data they contain.

PulseApp is a good example of how tables can be used to efficiently present and manipulate complex data.

Reports

Most businesses work with some kind of reports. Printed reports are usually required, so pay attention to the design of reports. Printed (or exported) reports are usually simplified versions of online reports, optimized for monochrome printers.

FreshBooks has printing, PDF exporting and “Send to email” features. It also shows a print preview.

Don’t Forget UI Design Patterns

We’re so used to hearing and talking about UI design patterns that we sometimes forget about them! UI design patterns are helpful for designing user interfaces. The important thing is to consider them early on in the design process, ideally at the sketching stage. Because patterns often solve common problems, the right pattern can facilitate the user’s familiarity with an interface and increase the speed at which they learn it.

This screenshot is from the article 12 Standard Screen Patterns, which goes over the most common screen patterns.

Further reading:

Designing Web Interfaces: Principles and Patterns for Rich Interactions
A fantastic book that covers more that 70 Web design patterns.
40+ Helpful Resources On User Interface Design Patterns

Case Study: Online Banking Application

To take an example from the real world, I will briefly explain the process of designing the user interface for one small bank’s online banking system. The team I worked with was hired to improve the system. The main reason for the redesign was that, according to management, “users complained and many stopped using it.”

After only a couple of hours spent with actual users, the main problems were uncovered. Information about accounts and credit cards was buried in poor navigation. Understanding how much money users were spending and the state of their accounts and credit cards was also hard. The application, however, was obvious to bank employees; they were familiar with the terminology and could interpreted the numbers in the application perfectly well.

Give the tight deadlines, we followed the process I have described, and we partially succeeded. Despite the short time, the major problems were so obvious that we clearly understood our main task and how to go about it. We created a dashboard that provided clear information on the state of all accounts and credit cards. With this new navigation, finding information became easier. Reports were easier to understand, and several new features were implemented.

Although we made only a few changes, the changes affected critical user tasks and resulted in significant improvements to the overall experience.

Final Thoughts

Designing user interfaces for business Web applications is a challenging job that is full of compromises. You have to make compromises between client and user needs; business requirements and users; novice and expert users; functionality and simplicity. It requires a solid understanding of users and their tasks, as well as of UI design principles and patterns. Despite the difficulties, the job is interesting, and you learn many new things on each project that influence the way you design websites.

While this article reflects some well-known concepts and things I have learned from designing business applications over the years, I look forward to hearing your experiences and stories.

(al)

The Future Of Nautilus

2010-02-28T23:57:30Z

Nautilus – the default file manager in Ubuntu – has an interface many consider to be awful and overly complex.

It has button after button, toolbar after toolbar, menu after menu – and just looks, in my opinion at least, clunky.

Ian “ Izo” Cylkowski also thinks that Nautilus could do with a makeover and has gone a step further and produced some mock-ups detailing his vision for the file browser.

Boy does it scream ‘Yes!!’

To get to his design, Cylkowski deconstructed the current Nautilus, highlighti ng its replicated buttons and superfluous menus.

It’s well worth reading the whole thing over at his site @ http://www.design-by-izo.com/2010/02/27/deconstructing-nautilus-and-rebuilding-it-better/ although a brief summary is below.

Arrows replicate functionality provided by pathbar. Would be more ‘elegant’ to have their feature merged into a right click on back/forward buttons.
‘Parent Folder’ button replicates functionality provided by pathbar.
Stop buttons is useless – file browsers load instantly.
Reload could be integrated into pathbar
Home button is already on sidebar.
Computer button should also be on sidebar.
Search should be a text entry field.
Make the pathbar sexy.
Zoom could be better implemented (like Nautilus Elementary’s slider)
Buttons would work better than drop down list; a-la OS X finder.

His resulting mock-up is worthy of an OMG!

He even goes one better by proposing that GNOME Activity Journal – which uses Zeitgeist to display a log of your recent file activity – would make more sense as part of Nautilus rather than a standalone application.

Seeing it in mock-up it’s hard to disagree!

Get it now (sorta.)

Cylkowski does reference the awesome work of DanRabbit’s Nautilus-Elementary. Better still Nautilus-Elementary can be installed today!

Install Nautilus-Elementary [Karmic only]

Open a terminal and type the following commands, hitting your return/enter key after each.

sudo add-apt-repository ppa:am-monkeyd/nautilus-elementary-ppa
sudo apt-get update
sudo apt-get upgrade

Then either log out and back in to use the new Nautilus or manually do it by pressing Alt+F2 and typing: -

pkill nautilus

Now we need to launch nautilus, so press Alt+F2 again, this time entering: -

nautilus

To hide/show the "menu bar" as in the screenshot hit Alt+M.
To toggle the sidebar use 'F9'.

Future of Nautilus-Elementay

DanRabbit’s vision for Nautilus-Elementary is not too dissimilar to that of Ian Cylkowski’s: -

We’ve already seen Nautilus-Elementary get CoverFlow-esque integration: -

And you needn't worry about losing the ability to 'type' in the pathbar either, as DanRabbit's vision has that covered in spectacular style: -

Practical future: Lucid, GNOME 3 and beyond

Leaving the land of mock-ups and 3rd party win, what IS Nautilus going to look like in the future?

Requiring almost no squinting in our crystal ball, Nautilus in Lucid already sees some minor, subtle changes.

Tabs have been moved to the bottom
Split-pane view has been implemented
Toolbars have been slightly rearranged with the ‘Places’ sitting in the pathbar and ‘View’ and ‘Zoom’ jumping up to the main toolbar.
Un-mounting a device now returns you to ‘Computer’

Split pane view in Nautilus 2.29

GNOME 3

GNOME 3 will see nautilus become solely a file manager. “What?” you ask. Currently Nautilus provides several parts of the GNOME desktop including your actual ‘desktop’. With GNOME 3 Nautilus will be relegated to simply being a file manager – making swapping it out for an alternative a breeze.

Although noting concrete has been built – or even decided on – yet last weeks GNOME usability hackfest in London threw up some tantalizing glimpses into what’s being proposed.

Behold the whiteboard of divination: -

Máirín Duffy posted the above image and some notes on the discussion of Nautilus from the hackfest on her blog - which I've shamelessly pillaged and pasted below.

Head on over to her blog for even more juice on GNOME 3 from the week-long discussion.

remove combo box to change nautilus view, just use menu?
frustration that nautilus is becoming like midnight commander, not beautiful to use, too complicated
dont like split pane idea, why not use two windows? split-pane seems universally disliked by GNOME designers.
snap to side-by-side like in windows 7 would be good and would remove need for split-pane.
search in nautilus sucks
confusion between magnifying glass icon for search vs zoom
Things to consider removing from ui:
- zoom controls
- tabs
Things to add to ui
- undo
- search folders
- sharing / collaboration – share with
- context-sensitive actions toolbar

Convergence

A lot of the discussions for GNOME 3’s direction tie in wonderfully with the proposed directions offered up by Elementary and Izo.

It seems the future of Nautilus has its bearing set firmly in the direction of awesome.

Thanks to leo

Introducing Read It Later Digest

2010-02-25T17:43:59Z

Digest is Read It Later with a brain. It analyzes all of the articles in your list and automatically groups them into topics. Simply put, it is like having folders in your list, but RIL takes care of sorting them for you.

This is all about reinventing the reading list. RIL makes it very easy to save a link with just one click, in any browser and on most mobile devices. It’s almost too easy. So naturally your reading list can fill up fast.

There is a fundamental problem with displaying your items in a list. It’s hard to know where to start. Do you start from the top? From the bottom? Randomly? None of these make a whole lot of sense because the order in which you saved things is not necessarily the order you want to read them.

Looking at your list, you may have an article about technology you wanted to read, a video about kittens you wanted to watch, a link to shoes you wanted to buy or a website for a band your friend told you to check out. And when you finally get around to reading, you may only be interested in one of those topics. It’d be a LOT easier if it were better organized.

Normally organization is accomplished with folders and tagging. But both of those take time to manually add to each item, time that you do not have, which is the whole reason why you are using Read It Later in the first place!

This is why Digest works. It’s smart enough to know the difference between an article on iPhones and an article about sports. It takes each article and filters it into the topics you care about automatically.

Landing on the front page is like coming to a news site where all of the articles were hand picked by you, where there is nothing to filter through, it is all content you already know you want to read.

Topics are automatically created for you based on what you’ve already saved but you can fully customize your digest by adding topics, removing them, rearranging them and even defining your own custom topics that articles will be sorted into.

Digest finds and highlights certain content from your list. If you save a lot of videos, Digest will present them together for you. If you save a lot of links from a certain site, Digest will group them for you.

You can even share your digest by making it public and sending the URL to all of your friends.

How to Get It

Today, Digest is available as a free invite only beta. To request an invite, go to http://readitlaterlist.com/digest

The Future

Digest is starting on the web but will very soon be making a native appearance within the iPhone app, Firefox extension and the upcoming iPad app.

The current way you use Read It Later and all that it offers will not change and it will absolutely still be free. Digest will be the premium upgrade to the Read It Later service and pricing will be announced in a few weeks.

I’m unbelievably excited about where Digest will go in the future. I already have a million ideas for paths to explore, so grab an invite, give it a spin and enjoy a new way to use Read It Later!

Why are you apologizing?

2010-02-25T10:02:00Z

I don't understand blog posts, emails and other messages that begin with an apology.

If you're sorry to interrupt me with that spam, don't send it.

If you know that yet another blog post on a topic that's not of interest to your readers will annoy them, don't post it.

If you're in HR and you know that no one in the office is going to read your office-wide spam about yet another inane meeting, don't bother us.

On the other hand, if it's important, if it needs to be said, if it benefits not just you but the recipient, then just send it. Instead of an apology, clearly label it so it's easy to ignore or discard. Even better, don't send everyone a message aimed at just a few people. It's easier than ever to focus on the people you need to focus on.

Just because it's more convenient for you to blast everyone in your address book doesn't mean it's smart.

Automatyczne sprawdzanie zgodności kodu Pythona z PEP8

2010-02-16T12:52:43Z

Jakiś czas temu pisałem o skrypcie który sprawdza zgodność kodu Pythona z PEP8. Jednak podczas pracy nad kilkoma projektami zauważyłem, że nie chciało mi się za każdym razem ręcznie uruchamiać tego skryptu na całym projekcie. Sprowadzało się do tego że uruchamiałem go gdy projekt był już skończony. Prowadziło do tego że nie byłem w stanie czasowo wprowadzić wszystkich poprawek, bo było ich bardzo dużo.

Niby wydawało by się że wystarczy raz czy dwa przeczytać dokument PEP8 i przy pisaniu kodu stosować się do zaleceń, jednak potem gdy pisze się kod to się o wszystkim zapomina. Wydaje mi się że z pisaniem kodu jest jak z ładnym odręcznym pisaniem, to nawyk którego się nie da się nauczyć czytając jakieś poradniki. Tak samo jest ze stylem pisania kodu.

Skrypt do sprawdzania poprawności stylu kodu sprawdza się tutaj znakomicie, pozwala w szybki sposób na wyuczenie nawyku pisania kodu zgodnego z PEP8. Aby ułatwić sprawę, dodałem do testów danego projektu, przypadek testowy który uruchamia skrypt i sprawdza kod całego projektu. Jeżeli kod nie jest zgody z PEP8, to wyrzuca wyjątek i pokazuje dokładnie co trzeba poprawić.

from StringIO import StringIO

import sys

import unittest

import pep8


class PEP8TestCase(unittest.TestCase):

    def test_pep8_rules(self):  
        sys.argv[1:] = ['--filename=*.py', '--show-source', '--show-pep8', 
                               my_project.__path__[0]]  
        buf = StringIO()         
        sys.stdout = buf
        pep8._main()            
        sys.stdout = sys.__stdout__   
        result = buf.getvalue()

        self.assertEqual("", result,
                     "Code messages should be empty but was:\n" + result)

Wystarczy teraz na bieżąco podczas pisania kodu uruchamiać testy i mamy pewność że powstały kod jest zgodny z PEP8. Bardzo fajnie sprawdza się to także z pisaniem kodu wykorzystując Test Driven Development. Podczas pisania i uruchamiania testów od razu mamy sprawdzony cały powstały kod.

Po paru poprawkach pisanie kodu zgodnego z PEP8 wejdzie nam tak w krew że podczas pisania kodu od razu zacznie się pisać poprawny kod. Przydaje się tutaj włączenie w edytorze pokazywanie białych znaków aby od razu wykrywać miejsca gdzie znajdują się nadmiarowe białe znaki.

Zauważyłem także, że dbając o styl kodu, wykrywa się też miejsca które można zrefaktoryzować i uprościć.

Google Buzz vs. Google Wave

2010-02-15T14:59:13Z

From the Google Wave Federation architecture white paper

Google Wave is a new communication and collaboration platform based on hosted documents (called waves) supporting concurrent modifications and low-latency updates. This platform enables people to communicate and work together in new, convenient and effective ways. We will offer these benefits to users of wave.google.com and we also want to share them with everyone else by making waves an open platform that everybody can share. We welcome others to run wave servers and become wave providers, for themselves or as services for their users, and to "federate" waves, that is, to share waves with each other and with wave.google.com. In this way users from different wave providers can communicate and collaborate using shared waves. We are introducing the Google Wave Federation Protocol for federating waves between wave providers on the Internet.

From a Buzz post by Dewitt Clinton, a Google employee

The best way to get a sense of where the Buzz API is heading is to take a look at http://code.google.com/apis/buzz/. You'll notice that the "coming soon" section mentions a ton of protocols—Activity Streams, Atom, AtomPub, MediaRSS, WebFinger, PubSubHubbub, Salmon, OAuth, XFN, etc.

What it doesn't talk much about is Google. That's because the goal isn't Google specific at all. The idea is that someday, any host on the web should be able to implement these open protocols and send messages back and forth in real time with users from any network, without any one company in the middle. The web contains the social graph, the protocols are standard web protocols, the messages can contain whatever crazy stuff people think to put in them. Google Buzz will be just another node (a very good node, I hope) among many peers. Users of any two systems should be able to send updates back and forth, federate comments, share photos, send @replies, etc., without needing Google in the middle and without using a Google-specific protocol or format.

From Mark Sigal’s post Google Buzz: Is it Project, Product or Platform?

I think that it's great that Google is iterating Gmail (read Tim O'Reilly's excellent write-up on it here), and actually improving an existing product, versus rolling out a knock-off of something that is already in the market.

Nonetheless. I am confused. I thought that Google Wave was destined to be the new Gmail, but after yesterday's announcement, I am left wondering if Gmail is, instead, the new Google Wave.

Since the saying goes that people in glass houses shouldn’t throw stones, I won’t make any comment besides sharing these links with you.

Now Playing: 50 Cent - Crime Wave

Where's the Money, Bob?

2010-02-08T13:08:40Z

The social web is starting to sound more and more like the dot.com days. We are in the middle of a transition. The ways we do things are radically different. What works and doesn't work seems new. And many people are getting very excited about numbers, but not the right ones.

Back in 2000, it was all about traffic. Just get a million people to come to your site, and you have made it. Today people say that attention, engagement, interactions or number of fans is the new currency. Just get people's attention and you have made it!

I have this weird sense of deja vu. Haven't we been here before? Didn't people learn anything?

The bottom line is this. Money is still the new currency. I can't buy a cup of coffee just by paying attention to it!

Should content be free?

Everywhere you look, you will find many people believing that content, and in some cases, products should be free. If you ask people about paying for news online, about 40-80% say that they won't. Content should be free, just as music should be free.

But here is the thing.

I am a firm believer of freedom of content. That is anyone should be free to get any content, despite where they are or how they choose to consume it. We need freedom of content, no restrictions, no lock-ins, no "you can only get it if you buy it on a DVD" or "you can only read this article in this news application, and not via RSS"

But that doesn't mean that centent itself should be free. It just means that consumption of content should be free.

Content creation is a different matter. Creating content cost money and time. When you spend 5 hours writing a really interesting article, you are creating value, and you do at a cost. You spend time on it. You have to buy a computer, have an internet connection, use electricity, and the list goes on and on.

Just take this site. It costs me $4,000/year to run this site just for at the equipment and connections alone. That doesn't even include the time I spend writing content, contacting companies, preparing graphics, and illustrations etc. If I had to hire a writer to do it for me, I would have to pay her an additional $36,000/year.

So here is a simple question. Would you pay for baekdal.com? And if so, how much? What about $9.99/year? Does that sound reasonable?

Notice: No, I am not planning to turn baekdal.com into a pay-site.

You know, Rupert Murdoch is right when he says that people need to understand that content costs money to make, and as such cannot be free. Free means somebody else have to pay the bills.

What Rupert Murdoch doesn't understand, though, is that he is selling the wrong product. People don't want cheap newspapers, they want highly targeted, uniquely created news. Rupert Murdoch tries to sell us a box of random news - who wants that?

Content consumption must be free. You must be able to consume content where ever you are, whenever you want, via whatever channel. People should never pay extra for a mobile version, or not be able to read it in their favorite newsreader.

But, the content and services themselves cannot be free. It costs money to create, thus comes at a price.

A Zero Rupees note... a way for people to pay nothing.

Notice: Also read: "How Newspapers Can Charge for Content"

Freemium doesn't mean free

The biggest problem with this "everything must be free" is actually caused by lot of people and companies not understanding the concept of freemium.

Freemium is a business model brilliantly explained by Chris Anderson (Editor of Wired Magazine) and it goes like this: I will give you this for free, so that you in turn might be persuaded to buy my other products for money. Or variation of the same.

It works for all kinds of industries. Online services might provide a free basic account, in the hope that you will later sign-up for premium services.

Or, the most used freemium model of all. I Will give you free content, in exchange for being exposed to advertisement. This is how almost every commercial website operates.

When people and companies starts to believe that freemium means free then we got real a problem. Freemium doesn't mean free. It means we will give you a salad, so that you might buy a meal to go with it. Some people might only want the salad, and that's okay, but, the business model is to get people to by the meal.

Most new social startups doesn't seem to understand freemium at all. Just take almost every single twitter app ever made. They are all free, with no viable business plan, besides hoping that huge amount of traffic will somehow make the millionaires.

How long do they plan to live of venture capital funding, before being sold to a clueless media company or go bankrupt? Doesn't this remind you of the good old dot.com days?

The other and much bigger problem is when the money part doesn't cover the expenses of running the free part. This is currently the biggest issue for newspapers. For traditional media companies like the New York Times, advertising online doesn't cover the cost of writing the articles.

Seesmic got the same problem. Their business model is to build huge amount of traffic, then introduce advertising, and make billions - or so they hope. But, because the market they are in is so crowded, the second they put advertising into their product, people will simple use another Twitter client.

It is a recipe for failure.

The coming battle

There is a war coming, and it will be nasty ...and the best thing that ever happened.

The war is about companies realizing that the internet is now their primary business, their primary communication channel, and their primary sales channel. And it has to make to make a profit, otherwise, they are sunk!

It is the war against the "everything must be free" culture, and it is won by the ones who create something worth paying for.

It is not about traditional companies trying to go back to the way it was (they are merely a distraction). It is not about advertising. And it is not about getting people to cover the cost of print.

It is about giving people a salad that is so good, that they will upgrade it to get a premium meal to go with it.

- Read Comments

influencer stalking

2010-02-07T16:32:19Z

I doubt there's a marketing plan in the world nowadays without some form of influencer marketing focus. Reaching influencers is part of every agency pitch. With mainstream media becoming less effective over time and widespread recognition that influencers play a big role in buying decisions, the influencer hunt is on.

The challenge is how to do it right. How do you identify who is really an influencer? How do you reach them? And with everyone else targeting influencers, how do you break through the clutter?

Many brands apply the same heavy-handed approach to influencer marketing that they do with mainstream marketing. They shout.

Somehow I ended up on agency influencer lists this last year, and I now receive dozens of emails a week, asking me to write about some piece of news or another. Aside from the fact that these emails are unsolicited, they are usually badly targeted, often lacking any news that is even moderately relevant to what I cover. The other problem is that they usually arrive in an impersonal press release format with "for immediate release" marked at the top. One recent note was addressed to "Dear <first name> <last name>" because their email mail merge didn't work.

It's a common pitfall to assume that influencer marketing is just the same old marketing, only targeted more finely. If whatever you're trying to share is mediocre from the start, it's simply not going to spread, no matter how loudly you shout it or how many people you put on your shouting list. It doesn't matter if they're influential or not.

One of my favorite (albeit 5-year old) examples of a bootstrapped influencer marketing program is the Stormhoek winery 100 Blogging Dinners in 100 Days, developed by Hugh MacLeod. The winery credits this event for doubling their sales in 12 months. They sent out a free pre-release bottle of wine to any blogger who was interested (with no obligation to write about it), along with a great manifesto about wine and blogging. It didn't shout. It started an intriguing conversation and talked to bloggers as people, not as press contacts or direct marketing targets.

I like this quote from Hugh after the event: "Blogging as a marketing tool is easier when you think of it as a chemical catalyst, not as a hammer and nail." I think that "chemical catalyst" is a good description of influencer marketing in general too.

Jakiego języka programowania się nauczyć?

2010-02-10T00:37:53Z

Ostatnio coraz częściej spotykam się z pytaniem osób, które chcą się nauczyć programować, jakiego języka warto się nauczyć pod kątem przyszłego rynku pracy. Czy jeżeli nauczę się języka X to będą miał pracę? Co lepiej, Y czy Z, żeby dużo zarabiać? I tego typu różne...

Minister, Ernst & Young

2010-02-06T21:33:10Z

Ministerstwo Nauki i Szkolnictwa Wyższego ogłosiło wczoraj Strategię Rozwoju Szkolnictwa Wyższego w Polsce do roku 2020 . Strategię , na zlecenie Ministerstwa, opracowała firma doradcza Ernst & Young, za 1,7 mln złotych . Dokument ten, omawiany już na wielu forach, zawiera szereg rekomendacji. Część z nich jest lepsza, część gorsza, ale kilka budzi mój sprzeciw:

Zakaz zatrudniania własnych doktorów. Uczelnie publiczne przez dwa lata nie będą mogły zatrudniać osób, które uzyskały na nich doktorat. Ma to wyeliminować "chów wsobny" tudzież wymusić mobilność. Mobilność to słowo-klucz. Nie możesz być dobrym naukowcem, jeśli nie otarłeś się o inne uczelnie. To prawda, tak to działa na przykład w Ameryce, ale autorzy Strategii nie wzięli pod uwagę ograniczeń, warunków brzegowych. Gdzie i za co młody doktor miałby się przeprowadzić? Za co wynająć mieszkanie, gdy pensja młodego doktora nie wystarcza na wynajęcie czegoś w dużym mieście po cenach rynkowych? Co ze współmałżonkiem doktora? W Polsce społeczeństwo w ogóle jest mniej mobilne, niż w Ameryce, wyjazd do innego miasta za pracą - i to nie ze wsi, ale z miasta uniwersyteckiego! - jest nam kulturowo obcy. W porządku, być może trzeba to zmienić, uczelnie powinny być otwarte na osoby z zewnątrz, ale nie sądzę, żeby administracyjny nakaz był najlepszą metodą. Duże dyscypliny, uprawiane na wielu uczelniach, szybko znajdą sposób na obchodzenie zakazu: nasz doktorant formalnie będzie się bronił w Instytucie Fizyki Jądrowej, na AGH lub w Katowicach, doktorant z Katowic u nas. Przecież nie ma prawnego obowiązku bronienia się na uczelni, na której odbywało się studia. A jeśli takiej możliwości nie będzie, młodzi doktorzy mogą zostać zmuszeni do porzucenia dalszej kariery akademickiej.
Kandydat na profesora musi przepracować łącznie co najmniej trzy lata poza własną uczelnią. Znów, w teorii pomysł jest znakomity, przyszły profesor musi mieć obycie w świecie, musi zetknąć się z innymi projektami i inną kulturą naukową, musi wystawić się na krytykę "obcych". Ach, żeby tylko pani minister mogła administracyjnie zapewnić miejsca pracy dla kandydatów na profesorów! W rzeczywistości może dojść do tego, że klasyczny wieloetatowiec będzie spełniać ten warunek - przecież pracował poza swoją uczelnią - podobnie jak osoba, która nigdzie długo nie zagrzała miejsca, i to niekoniecznie ze względu na ciekawość intelektualną pchającą ją do nowych wyzwań, a ktoś, kto pracował w jednym miejscu, nawet na światowym poziomie, będzie bez szans. Albo kandydat na profesora będzie musiał pogodzić się z czasowym spadkiem dochodów i prestiżu, popracować pewien czas w gorszej uczelni. Czego on się tam nauczy? I czy będzie miał gdzie pracować, bo przecież Strategia zakłada konsolidację i likwidację małych uczelni, także ze względów demograficznych. Oj, chyba nie zostanę profesorem. O tym, czy ktoś na stanowisko profesora zasługuje, powinny decydować nie formalne kryteria administracyjne, ale uzyskane wyniki - publikacje, zaproszone wykłady na konferencjach. (Oj, chyba nie zostanę profesorem.) Strategią obronną będzie mianowanie na stanowiska profesorów podwórkowych, pardon, uczelnianych, wszystkich doktorów habilitowanych tuż przed wejściem reformy w życie. Obawiam się, że oba te sposoby na administracyjne wymuszenie mobilności będą w wielu wypadkach działać raczej zniechęcająco niż mobilizująco.
Zakaz wieloetatowości - pracownik uczelni akademickiej będzie mógł poza własną uczelnią pracować co najwyżej na pół etatu. Dotyczyć ma to też działalności gospodarczej. Świetny pomysł, który mnie zresztą powinien być obojętny - fizyków nikt nie potrzebuje, więc ich na drugim itd etacie nie zatrudnia. Takie obostrzenia są zrozumiałe, o ile prestiż i wynagrodzenie pracownika uczelni są na tyle duże, żeby rekompensować utratę potencjalnych zarobków gdzie indziej. Tymczasem na sama strategia zakłada, że do roku 2020 realne wynagrodzenia pracowników uczelni mają wzrosnąć o 30%. To zbyt mało. Ha, już widzę naszych licznych profesorów prawa ( nomina sunt odiosa ), którzy porzucają swoje kancelarie adwokackie, żeby móc pozostać na uczelni. Podobnie pracowników wydziałów medycznych, porzucających swoje gabinety. Nawiasem mówiąc, pracownicy wydziałów medycznych bardzo często mają tylko cząstkę etatu akademickiego, w ramach którego uczą i prowadzą badania naukowe, oraz cały, lub też jakiś ułamek, etatu z PSK, w ramach którego leczą ludzi. Czy teraz będą musieli przestać leczyć, czy też będą leczyć za darmo? Czy wysoko opłacanym specjalistom z E&Y w ogóle to przyszło do głowy?
Likwidacja stypendiów naukowych. Środki przeznaczane obecnie na stypendia naukowe pójdą na zwiększenie liczby i wysokości stypendiów socjalnych, aby zwiększyć dostępność darmowych studiów. Stypendia mają być przyznawane nie przez uczelnie, ale przez Ośrodki Pomocy Społecznej. Ubóstwo nie powinno być barierą do studiowania, powinny być liczne programy stypendialne wspierające niezamożnych, ale zdolnych studentów. No właśnie, zdolnych. Po wejściu w życie ministerialnych propozycji takie samo wsparcie otrzyma student biedny, ale zdolny i pracowity, jak i biedny leniwy kombinator, któremu zależy tylko na tym, żeby utrzymać się na studiach, bo dzięki temu dostaje stypendium. To jest po prostu głupie. I jak ma się to do deklarowanego celu, jakim ma być podniesienie i poziomu wykształcenia, i jakości badań? Ano, nijak.
Ostateczna likwidacja jednolitych studiów magisterskich. Tak zwany "proces boloński", podział studiów na studia I i II stopnia, uznanie studiów doktoranckich za studia III stopnia, to katastrofa, zupełnie nieprzystająca do polskich warunków i tradycji (pracodawcy osoby z samym licencjatem traktują jak ludzi z niepełnym wykształceniem!), ale OK, jakoś sobie z tym radzimy. Ale teraz co, będzie licencjat z prawa? Licencjat z medycyny?! Przywracamy zawód felczera?
Wprowadzenie kategorii uczelni badawczej . Uczelnia badawcza ma prowadzić badania, zatrudniać personel pod kątem ich pozycji naukowej, nie zaś potrzeb dydaktycznych, ma obowiązek prowadzić studia III stopnia i może prowadzić studia I i II stopnia, na uczelni badawczej na jednego pracownika naukowego ma przypadać mniej studentów, niż na uczelni akademickiej. Jak rozumiem, mają się z tym łączyć większe pieniądze na badania. Nie wiem, jakie będą szczegółowe kryteria, ale jeśli mały stosunek liczby studentów do pracowników potraktować literalnie jako wymóg, wielkie uczelnie - Uniwersytet Warszawski, Jagielloński i inne czołowe uniwersytety - prowadzące masowe, popularne kierunki studiów, nie będą mogły stać się uczelniami badawczymi. Za to instytuty naukowe, które prowadzą studia doktoranckie, będą mogły formalnie zjednoczyć się w uniwersytet, uzyskać status uniwersytetu badawczego i spić całą śmietankę. Kudrycka jako eksponent lobby instytutów naukowych?
Zwiększenie kontroli ministra nad uczelniami państwowymi. Tak, duże uczelnie państwowe są źle zarządzane, ale pomysł, że scentralizowana kontrola jest najlepszym remedium, to jest etatystyczna aberracja. Uczelnie, także państwowe, będą mogły upadać (w sensie prawa upadłościowego). Czy to ma oznaczać, że uczelnie staną się przedsiębiorstwami nastawionymi na zysk? W ogóle różnica pomiędzy uczelniami państwowymi a prywatnymi zaciera się - tyle, że uczelnie prywatne będą miały większą autonomię, niż państwowe. Minister Kudrycka jednak jest eksponentem interesów uczelni prywatnych.

Jednocześnie pani minister Kudrycka opublikowała artykuł , w którym zachwala swoją strategię. Artykuł jest cukierkowo niekonkretny, pełno w nim haseł, pobożnych życzeń, ale żadnych gwarancji. I jeszcze te stereotypy: Natalia - kobieta - to "dociekliwa humanistka", która "skoncentruje się na psychologii społecznej [...] i intensywnej nauce języków obcych", Wojtek - mężczyzna - to "umysł ścisły i praktyczny", który wobec tego pójdzie na politechnikę. Mnie zaś uderza wizja pani minister: uniwersytety to humanistyka, kierunki ścisłe to politechnika, która "musi zbliżyć swoją ofertę dydaktyczną do oczekiwań przedsiębiorców", są jeszcze wyższe szkoły zawodowe. A gdzie badania podstawowe , od matematyki po biologię? W wizji pani minister najwyraźniej nie ma na nie miejsca.

Mam nadzieję, że ten projekt reform nie wejdzie w życie.

100 Things You Should Know About People: #18 — What People Look At On a Picture Or Screen Depends On What You Say To Them

2009-12-11T22:21:03Z

Yarbus Eye Tracking Picture

Eye tracking is a technology that allows you to see and record what a person is looking at, and for how long. One way it is used is to study web sites to see where people are looking on a web page, where they look first, second, etc. It’s a pretty interesting technology, one of the benefits being that you don’t have to rely on what people SAY they are looking at, but can collect the data directly. Like any technology, however, it’s not perfect, and one of the problems with eye tracking is that you can’t just give people a web site to look at and then assume that where they look is what they are “really interested” in.

We underestimate the effect our instructions have on where someone looks. Look at the picture at the beginning of this post. In research by Yarbus, people were shown this picture, and then given different instructions of what to think about while looking at the picture. Below are the eye gaze patterns matched with the instructions that people were given:

Proceed with caution. To me this data says: a) If you are using eye tracking as a technique to evaluate how people are using your website then you must be very careful about the instructions you give, and you must make sure you are giving everyone the exact same instructions. b) You can’t assume that just because people look at one spot on your website when they first see it that they will always look there. It might depend on what they were coming back to do. c) It’s nice to have a measure that doesn’t rely on what the user says or how they think they are reacting, but even these “objective” measures aren’t as objective as we think!

This research is from way back, but I believe it is still relevant. I haven’t found any recent replication of it yet. If you know of any please do pass on the reference:

Yarbus, A. L. (1967). Eye Movements and Vision (B. Haigh, trans.), New York: Plenum

——————————————-

Did you find this post interesting? If you did, please consider adding your comment, subscribing to the blog via RSS or email, signing up for the Brain Lady newsletter, and/or sharing the post.