Category: SAS

50 million illegal aliens apprehended in the US

There’s been quite a bit of controversy about the number of undocumented immigrants in the US lately – for example, Ann Coulter claims that number is 30 million, whereas others claim it’s about 11 million (readers of my blog are data-savvy, and would dig into the details of such claims, […]

The post 50 million illegal aliens apprehended in the US appeared first on The SAS Training Post.

Deploy edx spark environment to DigitalOcean

This summer I took the Spark courses at edx CS100 and CS190, and had wonderful experience.
The two classes apply a Vagrant virtual machine containing Spark and all teaching materials. There are two challenges with the virtual machine —
  1. The labs usually take long time to finish, say 8-10 hours. If the host machine is closed, the RDDs will be lost and the pipeline has to be run again.
  2. Some RDD operations take a lot computation/communication powers, such as groupByKey and distinct. Many of my 50k classmates complained about the waiting time. And my most used laptop is a Chromebook and doesn’t even have options to install Virtual Box.
To deploy the learning environment to a cloud may be an alternative. DigitalOcean is a good choice because it uses mirrors for most packages, and the network speed is amazingly fast that is almost 100MB/s (thanks to the SSD infrastructure DigitalOcean implements for the cloud, otherwise the hard disk may not stand this rapid IO; see my deployment records GitHub).

I found that a Linux box with 1 GB memory and 1 CPU at DigitalOcean that costs 10 dollars a month will handle most labs fairly easy with IPython and Spark. A 2 GB memory and 2 CPU droplet will be ideal since it is the minimal requirement for a simulated cluster. It costs 20 dollars a month, but is still much cheaper than the cost to earn the big data certificate that is $100 (50 for each). I just need to write Python scripts to install IPython notebook with SSL, and download Spark and the course materials.

  • The DevOps tool is Fabric and the fabfile is at GitHub.
  • The deployment pipeline is also at GitHub

SAS Enterprise Guide now updates itself

I returned to work from a 2+ week vacation this morning. When I fired up SAS Enterprise Guide (as I do each work day and occasionally on weekends), I was greeted with this message: As a SAS insider, I knew this was coming. It’s a new feature that was added […]

The post SAS Enterprise Guide now updates itself appeared first on The SAS Dummy.

Was the dress blue … or was it teal, sky, turquoise, or spindrift?

I saw the dress photo as blue & black. If you’re a female, even if we perceived the exact same color, you might might not have said ‘blue & black’. That’s because women have a larger color vocabulary than men, and you might have elaborated on exactly which blue and […]

The post Was the dress blue … or was it teal, sky, turquoise, or spindrift? appeared first on The SAS Training Post.

Great Developers Communicate!

There are so many skills that make a difference between a good developer (someone who knows their syntax and has a bagful of good design patterns) and a great developer. Many of those differentiating skills are related to communication.

I wrote about good communication skills nearly six years ago, so it’s probably about time to revisit the subject!

Few if any of us work alone. The vast majority of us work as a team. Communication within the team is a necessity, not an option. As a developer, good communication will ensure you deliver the right thing and gain recognition for your abilities; as a manager, good communication is important if you and your team are to deliver what the business needs when the business needs it so that they can get maximum business benefit.

Many developers like working independently and are proud of the autonomy they may be given. But the paradox here is that the autonomy and independence can lead to a lack of recognition of their abilities. Many of us have worked damn hard only to see the boss give plaudits to somebody who may have contributed much less to the collective. Why does this happen, and how can we avoid it?

Surely there must be some hard numbers that will make it clear who’s contributing the most to the team. Well, in my experience, traditional metrics such as lines of code, bugs closed and features added all have drawbacks in the reality of day-to-day software development, e.g. are two small features more valuable than one larger feature? And clearly, softer activities such as writing-for-maintainability and helping/coaching others have no reliable metrics for comparison. So, if there are no reliable metrics, how do you get noticed?

I think it comes down to trust. When a manager gives autonomy and independence to team members they are trusting them to complete the assigned task, make wise and strategic decisions along the way, and pro-actively communicate problems long before the become a problem. For someone to invest their trust in us, we have to show that we are in fact a good investment. We might ask ourself:

Does my boss trust me?
Do my team-mates and peers trust me?
Have I done a good job to earn their trust?
How would my peers describe me to someone else?
How influential am I within the organisation?

As a team member, we want to be judged by our contributions, and we want autonomy and the ability to own substantial things. As a manager, we want to give recognition and praise to the people who deserve it, but we don’t want to micromanage and spend our days being Big Brother. So there’s an implicit contract: I will give you autonomy and independence, but it is your responsibility to share status and information with me.

For example, a team member once told me he had worked hard and really gave it his best, but from my viewpoint his progress wasn’t up to the same level of his team-mates. When he was leaving the company he told me all the things he had done – and I asked him “Why didn’t you share this with me before?” You see, I would have advised him to spend his time elsewhere on priorities that were more important to the business. He responded with “I thought you would know.” Don’t make that same mistake.

And so my conclusion in all of this is: if you want autonomy, and the ability to own and control your own domain and projects – it is your job to push information and build trust with your team members.

In other words, you need to learn and do the following:

Follow through. Do what you say and consistently deliver on your commitments.

Pro-actively communicate when a task takes you longer than you thought, and why.

Improve your communication skills. In order for others to hear you, sometimes you have to hone the way you deliver your message.

Volunteer information and make an effort to explain vague or hard to understand ideas and concepts. Make an effort to share the details of your decisions and diversions. This is also important when you make mistakes – letting others know before they figure out on their own will show ownership of the situation and can prevent misunderstandings later.

Be forthright and authentic with your feelings. Even when you may hold a contrary opinion communicate your thoughts (respectfully and with tact).

Don’t talk behind the backs of others. It is very difficult to build trust if someone knows that you will say something negative about your boss, the company leadership, or another team-member.

Be objective and neutral in difficult situations. Learn how to be calm under pressure and act as a diplomat resolving conflicts instead of causing them.

Show consistency in your behaviour. Not just in follow-through but by eliminating any double standards that may exist.

Learn to trust them. This is one of the hardest ones, but trust is a two-way street. Giving others the benefit of the doubt and learning how to work with them is essential to a strong mutual working relationship.

In turn, hopefully, you have a good manager that will be able to ask you good questions and take the time to understand your contributions. And if that is not your situation, then make sure you are sharing information with those around you; such as your peers, your boss, and other stakeholders.

Good leadership is keeping everyone on the same page, and if you want independence it is your responsibility to make sure people know what you are contributing.

I don’t claim to come close to following my own advice in all situations, but I do keep reminding myself of what I believe is the right route to trust and autonomy. What is your route?


Follow me on Twitter: @aratcliffeuk

How the Tour de France and SAS Factory Miner relate

July has been an exciting month for me. Not only because of the historic Tour de France this year… but even more because this month the new offering SAS Factory Miner was officially released! With SAS Factory Miner you can run predictive models in an automated model tournament environment to […]

The post How the Tour de France and SAS Factory Miner relate appeared first on The SAS Training Post.

Active Directory Authentication for SAS on Linux (with realmd)

This is another post in the series about configuring a SAS platform on Linux to use Integrated Windows Authentication (IWA), in this post I’m going to jot down some notes on steps 1-7 – configuring the Linux server for Active Directory (AD) Authentication. Some time has passed since I wrote the original post, and a […]

Presenting the Newest Member of The Little SAS Book Family

There is a new kid on the block: Exercises and Projects for The Little SAS Book Fifth Edition.  Rebecca Ottesen, Lora Delwiche and I have worked for three years to complete this book of multiple choice, short answer, programming exercises, and projects.  This book is designed to be used as a supplement to the fifth […]