Quantcast
Channel: Planet Apache
Viewing all 9364 articles
Browse latest View live

Aaron Morton: Meltdown's Impact on Cassandra Latency

$
0
0

What impact on latency should you expect from applying the kernel patches for the Meltdown security vulnerability?

TL;DR expect a latency increase of at least 20% for both reads and writes.

History

The Meltdown vulnerability, formally CVE-2017-5754, allows rogue processes to access kernel memory. Simple demonstrations have already appeared online on how to expose passwords and ssh private keys from memory. The consequences of this, in particular on shared hosts (ie cloud) are considered “catastrophic” by security analysts. Initially discovered in early 2017, the vulnerability was planned to be publicly announced on the 9th January 2018. However, due to the attention generated by the frequency of Linux kernel ‘page-table isolation’ (KPTI) patches committed late in 2017 the news broke early on 3rd January 2018.

Impact

Without updated hardware, the Linux kernel patches impact CPU usage. While userspace programs are not directly affected, anything that triggers a lot of interrupts to the CPU, such as a database’s use of IO and network, will suffer. Early reports are showing evidence of CPU usage taking a hit between 5% and 70%. Because of the potential CPU performance hit and lack of evidence available, The Last Pickle used a little time to see what impacts we could record for ourselves.

Target

The hardware used for testing was a Lenovo X1 Carbon (gen 5) laptop. This machine runs an Intel Core i7-5600U CPU with 8Gb RAM. Running on it is Ubuntu 17.10 Artful. The unpatched kernel was version 4.13.0-21, and the patched kernel version 4.13.0-25. A physical machine was used to avoid the performance variances encountered in the different cloud environments.

The Ubuntu kernel was patched according to instructions here and the ppa:canonical-kernel-team/pti repository.

Stress

A simple schema, but typical of many Cassandra usages, was used on top of Cassandra-3.11.1 via a 3 node ccm cluster. The stress execution ran with 32 threads. Running stress, three nodes, and a large number threads on one piece of hardware was intentional so to increase thread/context switching and kernel overhead.

The stress run was limited to 5k requests per second so to avoid saturation, which occurred around 7k/s. The ratio of writes to reads was 1:1, with reads being split between whole partitions and single rows. The table used TWCS and was tuned down to 10 minute windows, so to ensure compactions ran during an otherwise short stress run. The stress ran for an hour against both the unpatched and patched kernels.

ccm stress user profile=stress.yaml ops\(insert=2,by_partition=1,by_row=1\) duration=1h -rate threads=32 throttle=5000/s -graph file=meltdown.html title=Meltdown revision=cassandra-3.11.1-unpatched

Results

The following graphs show that over every percentile a 20%+ latency increase occurs. Sometimes the increase is up around 50%.

Meltdown Cassandra median

Meltdown Cassandra 95th

Meltdown Cassandra 99th

Meltdown Cassandra stats

 

The full stress results are available here.


Steve Loughran: Berlin Buzzwords: CFP with an offer of abstract review

$
0
0
Berlin Buzzwords CFP is open, which, along with Dataworks Summit in April, is going to make Berlin the place for technical conferences in 2018.
Berlin
As with last year, I'm offering to review people's abstracts before they're submitted; help edit them to get the text to be more in the style that reviewers to tend to go for.

When we review the talks, we look for interesting things in the themes of the conference, try and balance topics, pick the fun stuff. And we measure that (interesting, fun) on the prose of the submissions, knowing that they get turned into the program for the attendees: we want the text to be compelling for the audience.

The target audiences for submissions then are twofold. The ultimate audience is the attendees. The reviewers? We're the filter in the way.

But alongside that content, we want a diverse set of speakers, including people who have never spoken before. Otherwise it gets a bit repetitive (oh, no, stevel will talk on something random, again), and that's no good for the audience. But how do we regulars get in, given that the submission process is anonymous?

We do it by writing abstracts which we know the reviewers are looking for.

The review process, then, is a barrier to getting new speakers into the talk, which is dangerous: we all miss out on the insights from other people. And for the possible speakers, they miss out on the fun you have being a speaker at a conf, trying to get your slides together, discovering an hour in advance that you only have 20 min and not 30 for your talk and picking 1/3 of the slides to hide. Or on a trip to say, Boston, having your laptop have a hardware fault and you being grateful you snapshotted it onto a USB stick before you set off. Those are the bad points. The good bits? People coming up to you afterwards and getting into discussion about how they worked on similar stuff but came up with a better answer, how you learn back from the audience about related issues, how you can spend time in Berlin in cafes and wandering round, enjoying the city in early summer, sitting outside at restaurants with other developers from around Europe and the rest of the world, sharing pizza and beer in the the evening. Berlin is a fun place for conferences.

Which is why people should submit a talk, even if they've never presented before. And to help them, feel free to stick a draft up on google docs & then share with edit rights to my gmail address, steve.loughran@ ;  send me a note and I'll look through.

yes, I'm one of the reviewers, but in my reviews I call out that I helped with the submission: fairness is everything.

Last year only one person, Raam Rosh Hai, took this offer up, And he got in, with his talk How to build a recommendation system overnight! This means that so far, all drafts which have been through this pre-review of submissions process, has a 100% success rate. And, if you look at the video, you'll see its a good talk: he deserved that place.


Anyway, Submission deadline: Feb 14. Conference June 10-12.  Happy to help with reviewing draft abstracts.

Rohit Yadav: AWS::S3::ERRORS::ACCESSDENIED (ACCESS DENIED):

$
0
0

Paperclip S3 AccessDenied

Aws::S3::Errors::AccessDenied (Access Denied):

Assuming you are using AWS’s IAM and you created a dedicated User for these uploads.

If you get this error when trying to upload to S3, you need to assign this IAM User the “AmazonS3FullAccess” Policy.


Rohit Yadav: uninitialized constant Paperclip::Storage::S3::AWS

$
0
0

Set up Ruby on Rails with Paperclip and S3 using AWS SDK

Paperclip requires the following gems added to your Gemfile.

If your paperclip version is 5.1.0 then we are using 'aws-sdk' version 2.3.

gem 'paperclip'
gem 'aws-sdk', '~> 2.3'

or our paperclip version is 4.1.0 then we need to use 'aws-sdk' version < 2  (note: add version less than 2.0 otherwise you will get paperclip error)

gem 'paperclip'
gem 'aws-sdk', '< 2.0'


Run bundle install and restart the Rails server after modifying the Gemfile.

WE CAN FOLLOW THE LINK BELOW :-

https://github.com/thoughtbot/paperclip/issues/2388
https://coderwall.com/p/vv1iwg/set-up-ruby-on-rails-with-paperclip-5-and-s3-using-aws-sdk-v2

Bryan Pendleton: Public Equity 501

$
0
0

A short lesson, this time; perhaps our education is nearing completion and we are moving on to become graduate students?

Clearlake Capital Acquires Perforce Software

Clearlake Capital Group, L.P. (together with its affiliates, “Clearlake”) today announced that it has acquired Perforce Software (“Perforce” or the “Company”), developer of the industry’s most flexible, scalable and secure version control and collaboration platform, from growth equity investor Summit Partners. The Company will continue to be led by Janet Dryer, CEO, and Mark Ties, COO, who will both join the Board of Directors alongside Clearlake. Financial terms were not disclosed.

Justin Mason: Links for 2018-01-10

$
0
0

Justin Mason: Links for 2018-01-11

$
0
0
  • [1801.02780] Rogue Signs: Deceiving Traffic Sign Recognition with Malicious Ads and Logos

    Well, so much for that idea.

    We propose a new real-world attack against the computer vision based systems of autonomous vehicles (AVs). Our novel Sign Embedding attack exploits the concept of adversarial examples to modify innocuous signs and advertisements in the environment such that they are classified as the adversary’s desired traffic sign with high confidence. Our attack greatly expands the scope of the threat posed to AVs since adversaries are no longer restricted to just modifying existing traffic signs as in previous work. Our attack pipeline generates adversarial samples which are robust to the environmental conditions and noisy image transformations present in the physical world. We ensure this by including a variety of possible image transformations in the optimization problem used to generate adversarial samples. We verify the robustness of the adversarial samples by printing them out and carrying out drive-by tests simulating the conditions under which image capture would occur in a real-world scenario. We experimented with physical attack samples for different distances, lighting conditions, and camera angles. In addition, extensive evaluations were carried out in the virtual setting for a variety of image transformations. The adversarial samples generated using our method have adversarial success rates in excess of 95% in the physical as well as virtual settings.

    (tags: signsroad-safetyroadstrafficself-driving-carscarsavssecuritymachine-learningcomputer-visionai)

  • The Stress of Remote Working – Martin De Wulf – Medium

    There is a lot of good to say about remote working, and I see a lot of rabid defence of the practice. That said, I have been working remotely for a little more than 5 years now, and I now must acknowledge that it does not come without stress. This might come as a surprise for some, but in the end, I think that remote working has taken some toll on me over the last two years, especially when I went almost fully remote for a year.
    I have to say, I agree with this 100% — I spent a few years remote working full time, and by the end of it I was absolutely delighted to return to a mainly office-based job.

    (tags: businessworklifecodingteleworkingremote-workstressanxietymental-health)

  • Best way designing a GDPR compliant datalake using AWS services : aws

    interesting thread at Reddit

    (tags: gdprredditawstipsdesignservicesops)

Justin Mason: Links for 2018-01-12

$
0
0
  • google/highwayhash: Fast strong hash functions: SipHash/HighwayHash

    HighwayHash: ‘We have devised a new way of mixing inputs with AVX2 multiply and permute instructions. The multiplications are 32×32 -> 64 bits and therefore infeasible to reverse. Permuting equalizes the distribution of the resulting bytes. The internal state occupies four 256-bit AVX2 registers. Due to limitations of the instruction set, the registers are partitioned into two 512-bit halves that remain independent until the reduce phase. The algorithm outputs 64 bit digests or up to 256 bits at no extra cost. In addition to high throughput, the algorithm is designed for low finalization cost. The result is more than twice as fast as SipTreeHash. We also provide an SSE4.1 version (80% as fast for large inputs and 95% as fast for short inputs), an implementation for VSX on POWER and a portable version (10% as fast). A third-party ARM implementation is referenced below. Statistical analyses and preliminary cryptanalysis are given in https://arxiv.org/abs/1612.06257.’ (via Tony Finch)

    (tags: siphashhighwayhashvia:fanfhashinghashesalgorithmsmacgooglehash)

  • Brain Cells Share Information With Virus-Like Capsules – The Atlantic

    …a gene called Arc which is active in neurons, and plays a vital role in the brain. A mouse that’s born without Arc can’t learn or form new long-term memories. If it finds some cheese in a maze, it will have completely forgotten the right route the next day. “They can’t seem to respond or adapt to changes in their environment,” says Shepherd, who works at the University of Utah, and has been studying Arc for years. “Arc is really key to transducing the information from those experiences into changes in the brain.” Despite its importance, Arc has been a very difficult gene to study. Scientists often work out what unusual genes do by comparing them to familiar ones with similar features—but Arc is one-of-a-kind. Other mammals have their own versions of Arc, as do birds, reptiles, and amphibians. But in each animal, Arc seems utterly unique—there’s no other gene quite like it. And Shepherd learned why when his team isolated the proteins that are made by Arc, and looked at them under a powerful microscope. He saw that these Arc proteins assemble into hollow, spherical shells that look uncannily like viruses. “When we looked at them, we thought: What are these things?” says Shepherd. They reminded him of textbook pictures of HIV, and when he showed the images to HIV experts, they confirmed his suspicions. That, to put it bluntly, was a huge surprise. “Here was a brain gene that makes something that looks like a virus,” Shepherd says. That’s not a coincidence. The team showed that Arc descends from an ancient group of genes called gypsy retrotransposons, which exist in the genomes of various animals, but can behave like their own independent entities.* They can make new copies of themselves, and paste those duplicates elsewhere in their host genomes. At some point, some of these genes gained the ability to enclose themselves in a shell of proteins and leave their host cells entirely. That was the origin of retroviruses—the virus family that includes HIV.

    (tags: brainevolutionretrovirusesvirusesgenesarcgagproteinsmemorybiology)


Bryan Pendleton: Artemis: a very short review

$
0
0

Andy Weir had the debut novel sensation that, surely, every novelist dreams of: The Martian was a world-wide best-seller, stayed on the best-seller lists for almost two years, and was then adapted to become one of the top ten movies of 2015.

You can only imagine what a life-changing experience this must have been for a guy who spent 15 years writing novels while working full time.

Anyway, Weir is now back with his second novel: Artemis.

In various creative fields, people talk about the "sophomore slump", and it surely can't have been easy for Weir to figure out how he wanted to write his next book. I'm sure he was also feeling pressure from both his readers and his publisher to hurry up and deliver another book.

So he did.

Artemis is certainly not the book that The Martian was.

However, both as a standalone effort and as a companion piece, it is quite interesting.

And, as you should probably grow to expect from Weir, it's a rollicking roller-coaster adventure ride of a book.

But while The Martian was a book about humans who were in space, and wanted to get back to Earth, Artemis is a book about people who were on Earth, and have decided that they want to live in space.

Weir is very interested in the notion of what it would mean for humans to be living somewhere other than on Earth, which is indeed a fascinating thing to think about, and Artemis is of most interest when you look at it from that viewpoint.

Artemis, as it turns out, spends most of its time spinning tales of completely ordinary experiences that have much more to do with being human beings, than with being in outer space. Rather than being just a sterile laboratory occupied by scientists, as so many "outer space" books are, Weir's outer space civilization is full of everything that makes us human. There are bars, casinos, and night clubs; there are prostitutes, drug dealers, and smugglers; there are petty rivalries, dirty laundry, and double-dealing.

But, most of all, there are complex systems, and, as was true with The Martian, it is when dealing with interesting complex systems that Weir's book is at its most interesting (even if great literature it ain't):

He wiggled his hand. "That wasn't just you. There were a lot of engineering failures. Like: Why aren't there detectors in the air pipeline for complex toxins? Why did Sanchez store methane, oxygen, and chlorine in a room with an oven? Why doesn't Life Support have its own separate air partition to make sure they'll stay awake if the rest of the city has a problem? Why is Life Support centralized instead of having a separate zone for each bubble? These are the questions people are asking.

Moreover, as Weir observes, these aren't actually engineering questions at their root; they are questions about how we organize our societies, a question which is just as important and relevant in outer space as it is here on Earth:

"The next big step is taxes."

"Taxes?" I snorted. "People come here because they don't want to pay taxes."

"They already pay taxes -- as rent to KSC. We need to change over to a property-ownership and tax model so the city's wealth is directly tied to the economy. But that's not for a while."

She took off her glasses. "It's all part of the life-cycle of an economy. First it's lawless capitalism until that starts to impede growth. Next comes regulation, law enforcement, and taxes. After that: public benefits and entitlements. Then, finally, overexpenditure and collapse."

"Wait. Collapse?"

"Yes, collapse. An economy is a living thing. It's born full of vitality and dies once it's rigid and worn out. Then, through necessity, people break into smaller economic groups and the cycle begins anew, but with more economies. Baby economies, like Artemis is right now."

Although Artemis ultimately fails as a work of literature, it is promising as a hint of what Weir is interested in, and where he might go.

Humans in space is a fascinating concept, and thinking about it realistically, rather than in some fantastic sterile implausible laboratory fashion, is how we're going to get to a point where we're actually ready to have humans in space. Building space ships and sending people out in them is just an engineering problem, and we'll solve that, probably pretty soon. But economics, politics, crime, government? These are actually HARD problems.

Writing about them, thinking about them, sharing those ideas, is one way to make it real, and for that, if for nothing else, I enjoyed reading Artemis and will look forward to Weir's next work.

Ortwin Glück: [Code] Gentoo intel-microcode reorg

$
0
0
Gentoo has given more love to its sys-firmware/intel-microcode package. It's now easier than ever to update microcode automatically early on boot. This is a valuable alternative to shipping microcode within the BIOS. I mean honestly, who regularly checks for BIOS updates? Much easier by doing that through an ebuild.

The new USE flag initramfs now builds a cpio archive at /boot/intel-uc.img ready to be used by grub. In /boot/grub/grub.cfg:
menuentry 'Gentoo Linux 4.14' {
  root=hd0,1
  linux /boot/linux-4.14.12 root=LABEL=ROOT ro rootfstype=ext4 net.ifnames=0
  initrd /boot/intel-uc.img /boot/initrd.img
}
Note how the microcode intitramfs is simply prepended to the boot initramfs (initrd). A kernel that has microcode laoding support enabled will find it there, upload the microcode into the cpu and then discard the initramfs blob, and continue booting with the initrd.img. The first line in your dmesg output will show:
microcode: microcode updated early to revision 0x80, date = 2018-01-04

Justin Mason: Links for 2018-01-14

$
0
0

Aaron Morton: Analyzing Cassandra Performance with Flame Graphs

$
0
0

One of the challenges of running large scale distributed systems is being able to pinpoint problems. It’s all too common to blame a random component (usually a database) whenever there’s a hiccup even when there’s no evidence to support the claim. We’ve already discussed the importance of monitoring tools, graphing and alerting metrics, and using distributed tracing systems like ZipKin to correctly identify the source of a problem in a complex system.

Once you’ve narrowed down the problem to a single system, what do you do? To figure this out, it’s going to depend on the nature of the problem, of course. Some issues are temporary, like a dead disk. Some are related to a human-introduced change, like a deployment or a wrong configuration setting. These have relatively straightforward solutions. Replace the disk, or rollback the deployment.

What about problems that are outside the scope of a simple change? One external factor that we haven’t mentioned so far is growth. Scale can be a difficult problem to understand because reproducing the issue is often nuanced and complex. These challenges are sometimes measured in throughput, (requests per second), size (terabytes), or latency (5ms p99). For instance, if a database server is able to serve every request out of memory, it may get excellent throughput. As the size of the dataset increases, random lookups are more and more likely to go to disk, decreasing throughput. Time Window Compaction Strategy is a great example of a solution to a scale problem that’s hard to understand unless the numbers are there. The pain of compaction isn’t felt until you’re dealing with a large enough volume of data to cause performance problems.

During the times of failure we all too often find ourselves thinking of the machine and its processes as a black box. Billions of instructions executing every second without the ability to peer inside and understand its mysteries.

Fortunately, we’re not completely blind as to what a machine is doing. For years we’ve had tools like debuggers and profilers available to us. Oracle’s JDK offers us Java Flight Recorder, which we can use to analyze running processes locally or in production:

mission control

Profiling with flight recorder is straightforward, but interpreting the results takes a little bit of work. Expanding the list of nested tables and looking for obvious issues is a bit more mental work than I’m interested in. It would be a lot easier if we could visualize the information. It requires a commercial license to use in production, and only works with the Oracle JDK.

That brings us back to the subject of this post: a way of generating useful visual information called a flame graph. A flame graph let’s us quickly identify the performance bottlenecks in a system. They were invented by Brendan Gregg. This is also part one of a very long series of performance tuning posts, we will be referring back to it as we dive deeper into the internals of Cassandra.

Swiss Java Knife

The approach we’ll examine in this post is utilizing the Swiss Java Knife, usually referred to as SJK, to capture the data from the JVM and generate the flame graphs. SJK is a fantastic collection of tools. Aside from generating flame graphs, we can inspect garbage collection statistics, watch threads, and do a variety of other diagnostic tasks. It works on Mac, Linux, and both the Oracle JDK and the OpenJDK.

I’ve downloaded the JAR, put it in my $HOME/bin and set up a shell function to call it easily:

sjk (){
	java -jar ~/bin/sjk-plus-0.8.jar "$@"}

On my laptop I’m running a workload with cassandra-stress. I’ve prepopulated the database, and started the stress workload with the following command:

cassandra-stress read n=1000000

For the first step of our analysis, we need to capture the stack frames of our running Java application using the stcap feature of SJK. To do this, we need to pass in the process id and the file to which we will dump the data. The dumps are written in a binary format that we’ll be able to query later:

sjk stcap -p 92541 -i 10ms -o dump.std

Then we can analyze the data. If all we have is a terminal, we can print out a histogram of the analysis. This can be pretty useful on it’s own if there’s an obvious issue. In this case, we can see that a lot of time is spent in sun.misc.Unsafe.park, meaning threads are just waiting around, parked:

$ sjk ssa -f dump.std --histo
Trc     (%)  Frm  N  Term    (%)  Frame
372447  96%  372447       0   0%  java.lang.Thread.run(Thread.java:745)
309251  80%  309251  309251  80%  sun.misc.Unsafe.park(Native Method)
259376  67%  259376       0   0%  java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
254388  66%  254388       0   0%  org.apache.cassandra.concurrent.SEPWorker.run(SEPWorker.java:87)
 55709  14%  55709        0   0%  java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
 52374  13%  52374        0   0%  org.apache.cassandra.concurrent.NamedThreadFactory$$Lambda$6/1758056825.run(Unknown Source)
 52374  13%  52374        0   0%  org.apache.cassandra.concurrent.NamedThreadFactory.lambda$threadLocalDeallocator$0(NamedThreadFactory.java:81)
 44892  11%  44892        0   0%  io.netty.util.concurrent.DefaultThreadFactory$DefaultRunnableDecorator.run(DefaultThreadFactory.java:138)
 44887  11%  44887        0   0%  java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
 42398  11%  42398        0   0%  io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:409)
 42398  11%  42398        0   0%  io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858)
 42398  11%  42398        0   0%  io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:753)
 42398  11%  42398        0   0%  sun.nio.ch.KQueueArrayWrapper.poll(KQueueArrayWrapper.java:198)
 42398  11%  42398        0   0%  sun.nio.ch.KQueueSelectorImpl.doSelect(KQueueSelectorImpl.java:117)
 42398  11%  42398    42398  11%  sun.nio.ch.KQueueArrayWrapper.kevent0(Native Method)
 42398  11%  42398        0   0%  sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86)
 42398  11%  42398        0   0%  sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97)

Now that we have our stcap dump, we can generate a flame graph with the following command:

sjk ssa --flame-f dump.std > flame-sjk.svg

When you open the SVG in a browser, you should end up with an image which looks something like this:

sjk-flame-screen

If you open the flame graph on your machine you can mouse over the different sections to see the method call and percentage of time it’s taking. The wider the bar, the more frequent it’s present in the stacks. It’s very easy to glance at the graph to understand where the time is spent in our program.

This is not the only technique for generating flame graphs. Brendan Gregg has a long list of links and references I recommend reading at the bottom of his FlameGraph page. I intend to write a utility to export the SJK format to the format that Brendan uses on his blog as it’s a little nicer to look, has a better mouseover, supports drill down, and has a search. They also support differential flame graphs, which are nice if you’re doing performance comparisons across different builds.

I hope you’ve enjoyed this post on visualizing Cassandra’s performance using FlameGraphs. We’ve used this tool several times with the teams we’ve worked with to tune Cassandra’s configurations and optimize performance. In the next post in this series we’ll be examining how to tune garbage collection parameters to maximize throughput while keeping latency to a minimum.

Justin Mason: Links for 2018-01-15

$
0
0
  • The likely user interface which led to Hawaii’s false-alarm incoming-ballistic-missile alert on Saturday 2018-01-13

    @supersat on Twitter: “In case you’re curious what Hawaii’s EAS/WEA interface looks like, I believe it’s similar to this. Hypothesis: they test their EAS authorization codes at the beginning of each shift and selected the wrong option.” This is absolutely classic enterprisey, government-standard web UX — a dropdown template selection and an easily-misclicked pair of tickboxes to choose test or live mode.

    (tags: testinguxuser-interfacesfaileashawaiifalse-alarmsalertsnuclearearly-warninghuman-error)

  • The Death of Microservice Madness in 2018

    Quite a good set of potential gotchas, which I’ve run into myself, including: ‘Real world systems often have poorly defined boundaries’ ‘The complexities of state are often ignored’ ‘The complexitities of communication are often ignored’ ‘Versioning can be hard’ ‘Microservices can be monoliths in disguise’

    (tags: architecturedevopsmicroservicesservicessoacodingmonolithsstatesystems)

  • Do algorithms reveal sexual orientation or just expose our stereotypes?

    ‘A study claiming that artificial intelligence can infer sexual orientation from facial images caused a media uproar in the Fall of 2017. […] Michal Kosinski, who co-authored the study with fellow researcher Yilun Wang, initially expressed surprise, calling the critiques “knee-jerk” reactions. However, he then proceeded to make even bolder claims: that such AI algorithms will soon be able to measure the intelligence, political orientation, and criminal inclinations of people from their facial images alone.’ ‘In [this paper], we have shown how the obvious differences between lesbian or gay and straight faces in selfies relate to grooming, presentation, and lifestyle? — ?that is, differences in culture, not in facial structure. […] We’ve demonstrated that just a handful of yes/no questions about these variables can do nearly as good a job at guessing orientation as supposedly sophisticated facial recognition AI. Therefore?—?at least at this point?—?it’s hard to credit the notion that this AI is in some way superhuman at “outing” us based on subtle but unalterable details of our facial structure.’

    (tags: culturefacial-recognitionaipapersfacial-structuresexual-orientationlgbtcomputer-vision)

  • Shanzhai ?? China & its Contents

    As he drinks Sino-coffee for around RMB 10, Comrade X might well be wearing the latest ‘ZARE’ couture while watching the TV news streaming on his HiPhone.[2] Back in Guangdong, his girlfriend — a sales consultant at a small stall in one of Shenzhen’s many wholesale electronics markets — sports a ‘high-end replica’ ?? Louis Vuitton bag and makes a living selling ‘domestically produced’ ?? and ‘smuggled’ ?? smartphones. The imitation products that festoon the couple’s lives are part of ‘shanzhai ?? China’. Shanzhai, the word means roughly ‘mass-produced imitation goods’, has created a Chinese landscape that is littered with products derided by the media, Chinese and international, as ‘copycat’, ‘guerrilla counterfeits’ and ‘knockoffs’, all the work of thieves.[3] Those who feel that their intellectual property and copyright has been infringed by shanzhai producers describe the products as ‘rubbish’, ‘piracy in disguise’ and ‘hooligan’.[4] Regardless of such righteous outrage, shanzhai — the producers, the products and the mentality — continues to flourish as an essential, quasi-legitimate shadow dimension of the Chinese economy. And, in practical terms, shanzhai products give disenfranchised ‘non-consumers’ of the orthodox economy — that is, people who would like to own but can’t afford the ‘original’ products — cut-price access to high-end technologies, as well as offering aspirational shoppers consumer satisfaction.

    (tags: shanzaichinafakesconsumerismhiphonesmartphonescopycatknockoffsimitationconsumption)

  • Don Norman on “Human Error”, RISKS Digest Volume 23 Issue 07 2003

    It is far too easy to blame people when systems fail. The result is that over 75% of all accidents are blamed on human error. Wake up people! When the percentage is that high, it is a signal that something else is at fault — namely, the systems are poorly designed from a human point of view. As I have said many times before (even within these RISKS mailings), if a valve failed 75% of the time, would you get angry with the valve and simply continual to replace it? No, you might reconsider the design specs. You would try to figure out why the valve failed and solve the root cause of the problem. Maybe it is underspecified, maybe there shouldn’t be a valve there, maybe some change needs to be made in the systems that feed into the valve. Whatever the cause, you would find it and fix it. The same philosophy must apply to people.

    (tags: don-normanuxuihuman-interfacehuman-errorerrorsriskscomp.risksfailures)

Justin Mason: Links for 2018-01-16

Bryan Pendleton: Oh this is a shame

$
0
0

For the last few years my favorite web site had become The Awl.

And now, no more.

Bummer.

I hope all those EXTREMELY talented writers and editors find good new locations elsewhere.


Rohit Yadav: HOW TO CALL RAILS API FROM SIMPLE HTML PAGE.

$
0
0

Calling the rails API from simple HTML page.

Step 1:  Create new form in the HTML page.

index.html.erb

<form action="javascript:void(0)">
  <input type="text" name="first_name" id="first_name">
  <select name="resident" id="resident">
    <option value="">Select</option>
    <option value="true">true</option>
    <option value="false">false</option>
  </select>
  <textarea type="textarea" name="notes" id="notes"></textarea>
  <input type="file" name="image" id="image">
  <button type="button" name="submit" id="form_submit_api">Send Data</button>
</form> 
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.js" integrity="sha256-DZAnKJ/6XZ9si04Hgrsxu/8s717jcIzLy3oi35EouyE=" crossorigin="anonymous"></script>

Step 2:  Write the script for passing the form data and post the data content to the Controller action using ajax.

<script type="text/javascript">
    $("#form_submit_api").on("click", function(){
      var formData = new FormData();
      formData.append('image', $('input[type=file]')[0].files[0]);
      formData.append('first_name', $("#first_name").val());
      formData.append('resident', $("#resident").val());
      formData.append('notes', $("#notes").val());
      $.ajax({
        type: "POST",
        url: "http://localhost:3000/api/v1/users",
        data: formData, 
        processData: false,
        contentType: false,
        success: function(response) {
          window.location.href = window.location.href;
        },
        error: function(response) {
          alert(response.statusText);
        }    
      }); 
});   
</script>

Now passing the image and other form data in the FormData :-

David Reid: Flirting with the dark side…

$
0
0

Last month our 2010 27″ iMac stopped working. There was no drama, just a black screen where previously had been the High Sierra backdrop. A quick bit of investigation showed that it wasn’t as simple as a dead machine. The fans still blew and when turned off and on it made all the expected noises. There was just no picture. Plugging in an external monitor soon showed that the machine was usable and still functioning normally. After 7 years of service it seemed as if the LCD panel had finally failed.

The law of murphy stalks all such events and this time was no different. We’re in the midst of building a house so buying a replacement iMac wasn’t really an option. Looking at the pricing and specifications the offerings, none seemed like value for money – even if we had the money and wanted to splash out. However, as the machine was my wifes day to day workhorse, we would soon miss the abilities it offered. A solution was needed.

When I built my desktop computer 15 months ago I chose standard, well supported components. Given the travel my work entails it often sits idle, so a possible low cost solution would have been for my wife to use my machine. Her MacBook Air is very old and frankly not very pleasant to use for anything beyond mail and surfing the web while my laptop is more than capable. Of course, my desktop runs Windows and she is happier with the world of Apple. But maybe…

The Dark Side beckons…

Hackintosh is a term I’ve seen a lot, but when I last looked it was very complex and involved a lot of jumping through hoops and using very specific components. Looking again in 2017 revealed how far things had evolved. A lot of reviews and blogs talked about success dual booting a single, home built machine with OSX and Windows. They all mentioned the tonymacx86 website which has a lot of information, tutorials and downloads to help.
Using a monitor I found a suitable USB memory stick, registered with tonymacx86, downloaded the files and shortly afterwards had the USB stick ready for a try. I’ll be honest, I didn’t expect much when I plugged it and powered up my desktop. The black screen with the Apple logo was a surprise. The progress bar slowly filling in was an even bigger surprise but when I was asked to choose an installation language I was glad to be sitting. Perhaps this could work?

Needing somewhere to install OSX on my desktop I found an older disk and after moving some wiring around had it installed and ready to receive OSX. Powering up again and going through the installtion instructions from the tonymacx86 website proved to be childs play and it didn’t take long for the install to commence. The reboots took me a little by surprise, but the process ran without any issues and soon I was filling in usernames and viewing the High Sierra backdrop.
Was everything perfect? No. That would have been too much to expect but I was sitting wondering what I was missing as up to this point it had been too simple.
Continuing with the tonymacx86 installation instructions I ran the MultiBeast app. This presented the first questions that caused me to pause. What drivers did I actually need? Looking at the OSX Preferences app it was clear no network or sound card had been recognised, so drivers would be needed for those. Ticking the boxes that seemed applicable and installing the Clover bootloader all went as expected and I rebooted – to a black screen.

D’oh

This was, ironically, the same situation as the existing iMac – a responsive computer with a black screen. The fact it had worked previously meant it wasn’t a total roadblock and after some research I copied the USB stick EFI folder over the same folder on the installed bootloader. Rebooting rewarded me with a login screen and working network! The sound card had been recognised and was listed but as my sound is via the HDMI cable I still wasn’t hearing anything. Having a 4K monitor on my desktop has been great, but when running OSX it wasn’t great as the font size was far too small. Without a way to change the display font using the machine was mildly frustrating despite the change from the i3 2010 processor to the i7 2016 processor being very evident.

I tried a few things but wasn’t able to get the sound working via HDMI after trying a large number of different approaches outlined by people. The community seems to be gaining cohesion and the tools are certaiinly improving but I still found myself looking at version incompatibilities, outdated and incomplete instructions that were often filled with abbreviations that meant nothing to me. Clover is an interesting tool but isn’t as user friendly as many claim it to be.

Fixed!

We discovered a fix for the black screen on the iMac during this process. It was a bug related to the iMac goiong to sleep. Armed with this knowledge we were able to restore it to full working order, removing the need for me to continue. While I haven’t taken this any further when the iMac does eventually cease to be usable I will seriously consider building a Hackintosh. A quick costing with suitable components showed that it would save around £1000 over a comparable iMac, resulting in a machine that could be expanded and upgraded.

Justin Mason: Links for 2018-01-17

$
0
0
  • Boost your immunity: Cold and flu treatments suppress innate immune system

    The next time you feel a cold coming on, maybe what you really want is just a little teensy bit of innate immune suppression, not an immunity boost. Over-the-counter medications like ibuprofen and antihistamines should help you feel better. Meanwhile, sit back while your acquired B and T cells do the rest. And if you aren’t yet sick, stay up-to-date on your vaccines, including the yearly influenza vaccine. Most importantly, practice vigorous hand washing — after all, the skin is also a component of your natural defenses and one that actually can be enhanced by good hygiene. Take care of yourself by keeping a balanced diet, maintaining good sleep habits, and minimizing stress. These are interventions that have been shown to help keep your immune system at its best. These alone can “boost” your odds of staving off an infection this cold season.

    (tags: immunityhealthimmune-systemcoldsb-cellst-cellsflu)

  • Sarah Jeong’s hilarious Twitter thread on Bitcoin

    “People are sick of the Federal Reserve, sick of bailouts, sick of inflation. You know what we need? Internet money with the usability of PGP and the reliability of BART” and much, much more

    (tags: bitcoinfunnysarah-jeongcomedylolspgptwitterthreads)

  • How To Measure the Working Set Size on Linux

    A nifty metric:

    The Working Set Size (WSS) is how much memory an application needs to keep working. Your app may have populated 100 Gbytes of main memory, but only uses 50 Mbytes each second to do its job. That’s the working set size. It is used for capacity planning and scalability analysis. You may never have seen WSS measured by any tool (I haven’t either). OSes usually show you virtual memory and resident memory, shown as the “VIRT” and “RES” columns in top. Resident memory is real memory: main memory that has been allocated and page mapped. But we don’t know how much of that is in heavy use, which is what WSS tells us. In this post I’ll introduce some new things I’ve developed for WSS estimation: two Linux tools, and WSS profile charts. The tools use either the referenced or the idle page flags to measure a page-based WSS, and were developed out of necessity for another performance problem.
    (via Amy Tobey)

    (tags: via:amytobeymemorylinuxrsswssprocpsprocessesmetricsworking-set-sizeram)

Bryan Pendleton: Up, up, and away

Rohit Yadav: *** No MethodError Exception: undefined method 'split' for : "content-type": Symbol

$
0
0

When we upload the videos using carrier wave or paperclip then we mostly getting some content-type error like -

*** No MethodError Exception: undefined method 'split' for : "content-type": Symbol

Then we have to check our RVM Version if version is 

 ruby-2.2.1

WE NEED TO CHANGE OUR RVM:-
WE NEED TO USE :-


 ruby-2.3.1
Viewing all 9364 articles
Browse latest View live




Latest Images