Life-changing decisions are happening in the dark. Machine-learning algorithms now help make decisions from loan applications to cancer diagnoses. In France, they place children in schools. In the US, they can set credit scores and insurance rates, and decide the fate of job candidates and university applicants.
But these programs are often unaccountable. To arrive at their decisions, machine-learning algorithms automatically build complex models based on big data sets, so that even the people using them may not be able to explain why or how a conclusion is reached. They're a black box. Wired: Keeping AI Honest (not online)
Again, design ethics are the most important thing we are facing right now.