helveticode.comAllan Lei – Medium

Get started Open in app Allan Lei 36 Followers About Follow Sign in Get started Follow 36 Followers About Get started Open in app Jun 26, 2020 Custom Installing NVIDIA Drivers for GKE I was recently working with ffmpeg and NVIDIA T4 GPUs on GKE for a encoding pipeline. To get started with GPUs on GKE, the NVIDIA drivers need to be installed on the nodes . After installing, ffmpeg should be able to access NVIDIA GPU capabilities like nvenc , nvdec , yadif_cuda , etc. One of the filters we needed was scale which there is a GPU accelerated version called scale_npp . The problem Unfortunately, scale_npp produced corrupt video when used. This turns out that the drivers install with the daemonset provided by GKE is version 410.79 and has some problems with NVIDIA T4 GPUs. … Read more · 2 min read 26 3 Sep 7, 2019 Blur Out Videos with FFmpeg Or how to utilize filter_complex Read more · 4 min read 77 1 Jan 7, 2019 Compiling the SIMD version of python-rapidjson A recent task had me taking a look at alternative JSON libraries for the purpose of performance. One of them was python-rapidjson which offered support for SIMD. Compiling To get python-rapidjson to compile with SIMD, we need to define one of the SIMD macros , either RAPIDJSON_SSE2 , RAPIDJSON_SSE42 , or RAPIDJSON_NEON . The chosen flag would then need to be passed to pip during install via CFLAGS . Depending on the flag, you would have to pass some addition options. SSE2: CFLAGS="-DRAPIDJSON_SSE2=1" SSE4.2: CFLAGS="-DRAPIDJSON_SSE42=1 -msse4.2" One-liner to re-install the currently installed version as the SIMD version. CFLAGS="-DRAPIDJSON_SSE42=1 -msse4.2" pip -v install -force-reinstall -no-binary python-rapidjson $(pip freeze | grep python-rapidjson) Published in SWAG · Dec 9, 2018 Moving forward with Live Stream Video Delivery Many popular apps offer live stream as a feature integration like Twitch and Youtube Live. With the ever increasing users drawn towards video based services, as seen with the trending race to top Youtube subscribed , the technology behind video delivery is equally increasing in importance. While most service will integrate video as a VOD like much of Youtube or DouYin , the challenge lies with live video delivery. So what’s available? There are many solutions available and is dependent on use case, but there are mainly 2 options, RTMP and DASH/HLS. While HLS is an option and having the much larger market share compared… Read more in SWAG · 4 min read 79 Sep 16, 2018 Auto-labeling GKE Nodes for XFS support To use XFS with Persistent Volumes, the host node needs to have the command xfs_mkfile available so disks can be created and formatted. The problem comes when needing to do this on GKE where there are 2 OSs available, ubuntu which has xfsprogs installed by default and cos which does not. Also, before ubuntu was available, container-vm was the used for XFS, but required a separate install . The solution I took was to add a XFS support node label at node pool creation time using OS image ubuntu . While this sort of works, there are a couple problems. The need… Read more · 2 min read 1 Sep 15, 2018 Disabling Transparent Huge Pages in Kubernetes I’ve recently needed to revisit some of our deployments which were created in the earlier days of GKE where some useful features were not available. One component revisited was the disabling the kernel setting Transparent Huge Pages (THP) recommended for mongo and redis . The solution at the time was to use a Daemonset running a startup script with gcr.io/google-containers/startup-script:v1 . There are a couple of areas that could be improved hostPID and securityContext seemed excessive No checks if the setting actually changed gcr.io/google-containers/startup-script:v1 is a relatively large image Timing conflicts with pod scheduling hostPID and securityContext Instead of using hostPID and priviledged: true … Read more · 2 min read 45 1 Jan 28, 2018 Installing kubectl in a Kubernetes Pod Without creating custom images tl;dr Use empty volume, initContainers , and subPath to copy and mount kubectl . The Why I needed access to the kubernetes API from within a pod so that the pod can self label itself. For example, I am currently working with redis and redis-sentinel. When sentinel triggers a reconfigure script, I want the pod to re-label itself to role=master or role=slave . I didn’t want to create a custom redis image that includes kubectl as it would be another component to maintain. Also, what if I needed to work with other images requiring kubectl ? Seemed like alot of maintenance going the custom image route. The How First… Read more · 1 min read 21 3 Apr 6, 2017 Running cron in Docker the Easy Way There are some situations where you just need to run a simple single command cronjob. This is where the alpine Docker image comes in very handy. It comes with a simple yet flexible cron package via busybox . Single Line For very simple stuff, a single command using the default alpine image. Mounted Volume For more complex crontabs, mount a crontab file! Compiled Docker Image Compiling your own image with an entrypoint makes it cleaner to run. 34 2 Jan 1, 2017 Mounting XFS on GKE Update Sept 6 2017 : As of Aug 8 2017, GKE has released the ubuntu image to replace container-vm . This image has xfsprogs pre-installed. Use this instead. We had a mongo replicaset cluster setup in Google Container Engine that was using the default ext filesystem when creating PersistentVolumes . The recommendation by mongo is that the xfs should be used instead . The default image on GKE is gci , based on ChromiumOS, which does not have a straightforward method for installing packages for xfs . container-vm does have xfsprogs , but is not installed by default Custom images are not supported on GKE. There’s no standard/recommended… Read more · 2 min read 56 1 Show more Jul 13, 2016 Streaming your RaspberryPi Camera To start streaming camera video, we simply combine raspivid and ffmpeg by piping one into another. Prerequisites raspivid ffmpeg raspivid -t 0 -w 1280 -h 720 -fps 25 -g 75 -fl -o - | ffmpeg -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=44100 -i pipe:0 -c:v copy -c:a aac -strict experimental -f flv -f flv rtmp://ip:port/app/stream -t 0 : Continuously pull video from the camera -w 1280 -h 720 -fps 25 : Video stream of 720p25 -g 75 : I-frame interval at 1/s -o - : Pipe video to stdout -f lavfi -i anullsrc=channel_layout=stereo:sample_rate=44100 and -c:a aac -strict experimental : Since the RaspberryPi doesn't have a microphone, we can pipe… Read more · 1 min read Allan Lei The Adventures of Me About Help Legal Get the Medium app...