Joi Ito's Web

Joi Ito's conversation with the living web.

L1002738-Enhanced-NR-1.jpg Photolog L1002738-Enhanced-NR-1.jpg Fri, Aug 4, 01:46 UTC

Calligraphy of lotus in the mud
泥中乃蓮 emerging from my long-neglected Japanese calligraphy. The symbol on the bottom is my kao (花押) which is a kind of kanji signature derived from my first name Joichi (穰一). Fudepen on back of index card.
Thick Nhat Hahn, one of my favorite Buddhist monks​​, often said, "No Mud, No Lotus." This is very similar to the saying, deichuunohasu (泥中乃蓮, でいちゅうのはす), which translates to "lotus in the mud." In Buddhism, the mud symbolizes suffering and darkness, from which emerges the lotus flower. Without the mud, the lotus would not emerge. There are sutras and meditations where one imagines oneself as the seed of the lotus emerging out of the mud.

Recently I've been studying and practicing Japanese tea ceremony, and one of the key elements of the tea room and the ritual is to choose a hanging scroll, often with something written on it by a monk. In my group, I have started exchanging seasonal Zen sayings and proverbs before tea sessions as a way to study both tea and Japanese. I've also started practicing my Japanese handwriting and calligraphy, which is in an abysmal state.

This week's proverb was "泥中乃蓮" which is seasonal because this is the week that lotuses are to begin opening according to the Japanese seasonal calendar. (It looks like the lotus blooming at the temple next door is already over. I guess we need to adjust the calendar for climate change.)

As I repeatedly wrote the proverb in my slowly improving, long-neglected handwriting, the characters emerged from my brush like the lotus trying to grow out of the mud. Along with the characters emerged a resonance with my own life which feels like a lotus trying to emerge from the mud of the last few years. It is also a societal metaphor for our society trying to come together around a common purpose and harmony in the midst of a truly mud-like moment in history.

And with this vision, I start this morning with a new metaphor and image to meditate on as we attempt to emerge from this submergence.

Two photos of the same scene comapring HDR and non-HDR
A comparison

tl;dr

iPhone videos shot in High Dynamic Range (HDR) would look blown out when edited in Premiere Pro. (Newer iPhones shoot in HDR mode by default now.) This was screwing up iPhone-user YouTubers, including myself. There were tons of not-too-useful videos on how to work around this, including selling you plugins and LUTs. In February 2023, Adobe fixed this by adding tone mapping so most of these “fixes” are mostly no longer helpful.


More detail followed by a How-To with images:

Newer iPhones now support Higher Dynamic Range (HDR) video, which has a “larger color space” and allows whites to be whiter and a broader range of colors making videos more vibrant than standard monitors and videos in Standard Dynamic Range (SDR).

The problem is that not all cameras, editors and displays support HDR, and the tools are just starting to support HDR.

Color spaces are standardized to be consistent across devices. The common color space standard for video is Rec. 709, which is what Adobe Premiere uses as a default. There is a different color space called Rec. 2100, which is a larger color space that supports HDR, unlike Rec. 709. If you record with the HDR setting on the iPhone, it will record in Rec. 2100.

The problem was that if your timeline on Premiere Pro was set to Rec. 709 and you added a clip recorded in Rec. 2100, the images looked blown out and saturated because the colorspace was too big and didn’t “fit” inside Rec. 709. You needed to either “map” Rec. 2100 to Rec. 709 and shrink the color space to fit in Rec. 709 or edit the entire video in Rec. 2100 by setting the color space to Rec. 2100.

Some people got thrown off because if you tried to edit a Rec. 2100 sequence with a normal display setting (your computer is default sRGB which is the computer equivalent of Rec. 709), the Rec. 2100 images would look blown out and weird on your display (even though they are actually fine on the sequence.) To properly edit Rec. 2100 videos, you must set your display settings in Premiere to map or recognize the Rec. 2100 settings in preview mode.

Lastly, even if you set the sequence to Rec. 2100 and the preview in the edit to Rec. 2100, if the export is set to Rec. 709, you would end up with the same blown-out image in the exported file. So the key to doing a proper HDR video is to shoot with HDR on, make sure your sequence is set to Rec. 2100 and your export is set to Rec. 2100. If you want to edit and export in Rec. 709 (normal video color space), just make sure you set tone mapping on and set your sequence and export to Rec. 709.

Luckily, if you set tone mapping on, any videos you put into your sequence will automatically map to whatever color space you edit. Also, if you choose New Sequence From Clip, the sequence will properly default to the color space that your clip is in.


How To Post iPhone HDR Videos to YouTube or Vimeo

Screenshot 2023-05-19 at 6.10.14.png

Go to Settings->General on Premiere Pro and ensure that Display Color Management and Extended dynamic range monitoring are on. This is required to view Rec. 2100 HDR videos in Premiere Properly. If you don’t set these, they may look blown out when you try to edit them.

Screenshot 2023-05-19 at 6.35.51.png If you record a video with HDR on and examine it in QuickTime, for example, you should see the color space as Rec. 2100. (In this image: Transfer Function: ITU-R BT.2100 (HLG)) Import this into Premiere Pro 23.2 or later.

Screenshot 2023-05-19 at 6.10.30.png Right-click this clip and select New Sequence From Clip.

Screenshot 2023-05-19 at 6.10.41.png

Right-click the sequence in the project pane and select Sequence Settings.

Screenshot 2023-05-19 at 6.10.53.png

Observe that Working Color Space is Rec. 2100 HLG. (HLG stands for hybrid log-gamma.) The Video Previews Codec should be Apple ProRes 422 HQ. This is the compression standard (codec) that supports HDR. This shows that the sequence is the same setting as the HDR media. Depending on the source resolution, the size may be HD or 4K.

Screenshot 2023-05-19 at 6.12.57.png

Go to export, select QuickTime as the format, and ensure your color space is set to Rec. 2100 HLG. Although the Apple Default is Rec. 2100 HLG for iPhone HDR videos, for posting on the Internet, Rec. 2100 PQ is probably better.

Screenshot 2023-05-20 at 9.46.28.png

If you export this and upload to YouTube or Vimeo, they should both recognize that they are HDR and display with high dynamic range for users able to view them. You will see “HDR” on the settings gear. It can take a few minutes for Vimeo and YouTube to process the HDR part.

See my sample video on YouTube and Vimeo. See, for example, how much brighter the whites in the video are than the white of the web page if you are viewing on an HDR compatible display. (Embedding HDR didn’t seem to work for me.)

If you instead would like to post as a normal video without HDR…

Screenshot 2023-05-20 at 11.10.32.png

Ensure your Sequence’s color space is set to Rec. 709 and that Auto Tone Map Media is set on.

Screenshot 2023-05-20 at 11.08.27.png Then make sure that you select Rec. 709 in the Export settings.

Enjoy!

Joi-profile-YT-HQ.jpg

Since returning to Tokyo in July last year after 14 years, I’ve been immersing myself in web3. I’ve also been frantically catching up with everything I missed in Japan - the food, my friends, Japan’s mostly failed attempts at digital transformation and the new generation of Gen Z kids.

I set up the Center for Radical Transformation at the Chiba Institute of Technology; took a new role, Chief Architect, at the company I co-founded, Digital Garage; and advise various government agencies and industry groups. I also launched a bunch of media projects. I have a podcast, a book on web3, a TV show on TV Tokyo’s satellite network, and a YouTube channel. I’ve been focused on publishing and interacting in Japanese, but I thought I’d check in with the Anglosphere to give you an update and share this • Notion page • on web3 in Japan in English.

Japan is trying very hard to transform society in many ways, and it’s wonderful being back and contributing to the effort.

If you looked at how many people check books out of libraries these days, you would see failure. Circulation, an obvious measure of success for an institution established to lend books to people, is down. But if you only looked at that figure, you'd miss the fascinating transformation public libraries have undergone in recent years. They've taken advantage of grants to become makerspaces, classrooms, research labs for kids, and trusted public spaces in every way possible. Much of the successful funding encouraged creative librarians to experiment and scale when successful, iterating and sharing their learnings with others. If we had focused our funding to increase just the number of books people were borrowing, we would have missed the opportunity to fund and witness these positive changes.

I serve on the boards of the MacArthur Foundation and the Knight Foundation, which have made grants that helped transform our libraries. I've also worked over the years with dozens of philanthropists and investors--those who put money into ventures that promise environmental and public health benefits in addition to financial returns. All of us have struggled to measure the effectiveness of grants and investments that seek to benefit the community, the environment, and so forth. My own research interest in the begun to analyse the ways in which people are currently measuring impact and perhaps find methods to better measure the impact of these investments.

As we see in the library example, simple metrics often aren't enough when it comes to quantifying success. They typically are easier to measure, and they're not unimportant. When it comes to health, for example, iron levels might be important, but anemia isn't the only metric we care about. Being healthy is about being nourished and thus resilient so that when something does happen, we recover quickly.

Iron levels may be a proxy for this, but they aren't the proxy. Being happy is even more complicated; it involves health but also more abstract things such as feelings of purpose, belonging to a community, security, and many other things. Similarly, while I believe rigor and best practices are important and support the innovation and thinking going into these metrics when it comes to all types of philanthropy, I think we risk oversimplifying problems and thus having the false sense of clarity that quantitative metrics tend to create.

One of the reasons philanthropists sometimes fail to measure what really matters is that the global political economy primarily seeks what is efficient and scalable. Unfortunately, efficiency and scalability are not the same as a healthy system. In fact, many things that grow quickly and without constraints are far from healthy--consider cancer. Because of our belief in markets, we tend to accept that an economy has to be growing for society to be healthy--but this notion is misguided, particularly when it comes to things we consider social goods. If we examine a complex system like the environment, for instance, we can see that healthy rainforests don't grow in overall size but rather are extremely resilient, always changing and adapting.

There is more to assessing a complex system than looking at its growth, efficiency, and the handful of other qualities that can be quantified and thus measured.

As biologists know, healthy ecosystems are robust and resilient. They can tolerate reductions in certain species populations ... until they can't. Scholars in ecology and biology have tried to model the robustness and resilience of systems in an effort to understand how to build and maintain such systems. Scientists have tried to apply these models to non-biological systems like the internet and ask questions, such as "How many and which nodes can you remove from the internet before it stops functioning?" These models are different from the mathematics economists use. Instead of relying on aggregate numbers and formulae, they use network models of nodes and links to ponder dynamics among connections in the system, rather than stocks and flows of economies.

Maybe there is something to learn from biologists and ecologists--the people who study the complex and messy real world of nature--when philanthropists are thinking about how to save the planet. We know from ecology and biology, for instance, that monocultures and simple approaches tend to be weak and fragile. The strongest systems are highly diverse and iterate quickly. When the immune system goes to war against a pathogen, the body engages in an arms race of mutations, deploying a diversity of approaches and constant iteration, communication, and coordination. Scientists also are learning that the microbiome, brain, and immune system are more integrated and complex than we ever imagined; they actually understand and tackle the more complex diseases currently beyond our scientific abilities. This research is pushing biology and computational models to a whole new and exciting level.

Many diseases, just like all of the systems that philanthropy tries to address, are complex networks of connected problems that go beyond any one specific pathway or molecule. Obesity is often described as simply a matter of managing one's calories and consequently cast as a lack of willpower on the part of an overweight individual. But it is probably more accurately understood in the context of a global food system that is incentivized by financial markets to produce low cost, high-calorie, unhealthy, and addictive foods. Calorie counting as the primary way to lose weight has been a rule of thumb, but we are learning that healthy fats are fine while sugar calories cause insulin resistance, which often leads to diabetes and obesity. So solving the obesity problem is going to require much more than increasing or reducing any one single thing like calories.
It's our food system that is unhealthy, and one result is overweight individuals.

In such a complex world, what are we to do? We need respect for plurality and heterogeneity. It's not that we shouldn't measure things, but rather that we should measure different things, have different approaches and iterate and adapt. This is how nature builds resilient networks and systems. Because we as a society have an obsession with scale and other common measures of success, researchers and do-gooders have a natural tendency to want to use simple measures (as described in our blog post) and other "gold standards" to gauge the impact of the money spent and effort expended. I would urge us to instead support greater experimentation, smaller projects, more coordination and better communication. We should surely measure indicators of negative effects--blood tests to measure what may be going wrong (or right) with our bodies are very useful for instance.

We also need to consider that every change usually has multiple effects, some positive and others negative. We must constantly look for additional side effects and dynamically adapt whatever we do. Sticking with our obesity example, there is evidence that high fat, low sugar diets, generally known as ketogenic diets, are great for losing weight and preventing diabetes; the improvement can be assessed by measuring one's blood glucose levels. However, recent studies show that this diet might contribute to thyroid problems and if we adhere to one, we must monitor thyroid function and occasionally take breaks from it.

Coming up with hypotheses about causal relationships, testing them and connecting them to larger complex models of how we think the world works is an important step. In addition, asking whether we are asking the right questions and solving the right problems, rather than prematurely focusing on solutions, is key. Jed Emerson, who pioneered early attempts to monetize the economic value of social impact, makes the same point in his recent book The Purpose of Capital.

For the last 1,300 years, the Ise Shrine in Japan has been ritually rebuilt by craftspeople every 20 years. The lumber mostly comes from the shrine's forest managed in 200 year time scales as part of a national afforestation plan dating back centuries. The number of people working at Ise Shrine isn't growing, the shrine isn't trying to expand its business, and its workers are happy and healthy--the shrine is flourishing. Their primary concern is the resilience of the forest, rivers, and natural environment around the shrine. How would we measure their success and what can we learn from their flourishing as we try to manage our society and our planet?

It is heartening to see impact investors developing evidence-based methods to tackle the complex and critical challenges that face us. It's also heartening that capital markets and investors are supportive of investing, and in some cases even accepting reduced returns, in an effort to help tackle our big, complex challenges. We must, however, make changes in the way we fund potential solutions so that it supports a diversity of disciplines and approaches. That, in turn will require new methods of measurement and perhaps we can take advantage of some very old ones, such as the data from Shinto priests who have been measuring ice on a lake for resist oversimplification. If we don't, we risk wasting these funds or, even worse, amplifying existing problems and creating new ones.

Ethan Zuckerman thoughtfully and appropriately points out that one big missing question in my recent Wired piece on measuring philanthropic impact is whether some of this positive societal change should be in the hands of government instead of philanthropists. He correctly points out that since the Reagan/Thatcher era of the 80s, we've started shrinking the role of government and have started to see big philanthropists and the private sector being called on to do what government used to do. In a post from 2013, Ethan wonders why he doesn't have rail solution to his commuting problem from Western Massachusetts. He suggests that without government, things like railway system are difficult to fund - the market isn't the best solution for many social goods.

I think the idea about whether we should be doubling down on philanthropy or fixing government and increasing government resources is a great question and probably the right one. I think the idea of fixing the government and turning the corner on the privatization is a daunting idea, but something we need to discuss.