The only reason why I would think Walt would grow Lily of the Valley in his home is because they have small white flowers which may be pretty, depending on your aesthetic tastes. Maybe it was Skyler who bought them, but it is strange that they have that poisonous plant in their home, especially when they have a small baby.
@Paanhalim, you're right, it was irresponsible of Walt to leave that little identification tag inside the plant. What if Jesse saw it one day when he was at Walt's house? Either way, I think the card was left there on purpose so it could be shown at the end of that one episode, as the camera slowly zooms in to it. It was mainly for the audience's surprise.