Data Annotation

What is the best way to collect Datasets for Annotation?

 

Data is the foundation of all the AI projects and there are different ways to prepare datasets, including collecting through the internet or consulting an agency. So, what is the best way to get raw data for the AI Data Training process?

One suggested way to collect the train and test data is to visit various open labeled resources like Google’s Open Images and mldata.org or many other websites providing datasets for training in ML projects. These platforms supply you with an endless multitude of data (mostly in the form of images) to start your training process. 

Depending on what kind of datasets you’re looking for, you can divide it into these categories of:

  • Open Dataset Aggregators
  • Public government Datasets for machine learning
  • Machine Learning Datasets for finance & economics
  • Image datasets for computer vision

For a high-quality machine learning / artificial intelligence project, datasets for training is the top priority that defines the outcome of the project. For the qualified and suitable datasets, you can consider the following filters to find the most suitable ones.

 

Open Dataset Aggregators

The most common thing that you might be looking for when working on machine learning / artificial intelligence is a source of free datasets. Open dataset finders that you can use to browse through a wide variety of niche-specific datasets for your data science projects. You can find it in:

  • 1. Kaggle: A data science community with tools and resources which include externally contributed machine learning datasets of all kinds. From health, through sports, food, travel, education, and more, Kaggle is one of the best places to look for quality training data.
  • 2. Google Dataset Search: A search engine from Google that helps researchers locate freely available online data. It works similarly to Google Scholar, and it contains over 25 million datasets. You can find here economic and financial data, as well as datasets uploaded by organizations like WHO, Statista, or Harvard.
  • 3. OpenML: An online machine learning platform for sharing and organizing data with more than 21.000 datasets. It’s regularly updated and it automatically versions and analyses each dataset and annotates it with rich meta-data to streamline analysis.

 

Public government Datasets

For machine learning projects concerning social matters, public government datasets are very important. You can find useful datasets in these following sources:

  • 4. EU Open Data Portal: The point of access to public data published by the EU institutions, agencies, and other entities. It contains data related to economics, agriculture, education, employment, climate, finance, science, etc.
  • 5. World Bank: The open data from the World Bank that you can access without registration. It contains data concerning population demographics, macroeconomic data, and key indicators for development. A great source of data to perform data analysis at a large scale.

 

Machine Learning Datasets for finance & economics

The use of machine learning / artificial intelligence for finance & economics has long been very promising with the vast implementation in algorithmic trading, stock market predictions, portfolio management, and fraud detection. The quantity for this is very big thanks to the datasets built over many years. You can find the easily accessible datasets for finance & economics here:

  • 6. Global Financial Development (GFD): An extensive dataset of financial system characteristics for 214 economies around the world. It contains annual data which has been collected since 1960.
  • 7. IMF Data: International Monetary Fund publishes data related to the IMF lending, exchange rates, and other economic and financial indicators.

 

Image datasets for computer vision

Medical imaging, automatic cars/self-driving cars are becoming more popular these days. With the high-quality datasets of training visual data, the application of these technologies will be better than ever. You can find the sources here:

  • 8. Visual Genome: A large and detailed dataset and knowledge base with captioning of over 100.000 images.
  • 9. Google’s Open Images: A collection of over 9 million varied images with rich annotations. It contains image-level label annotations, object bounding boxes, object segmentation, and visual relationships across 6000 categories. This large image database is a great source of data for any data science project.
  • 10. Youtube-8M: A vast dataset of millions of YouTube video IDs with high-quality machine-generated annotations of more than 3,800 visual entities. This dataset comes with pre-computed audio-visual features from billions of frames and audio segments.

Finding the suitable datasets for machine learning / AI is never easy. Besides the 4 categories mentioned above, the datasets can be Natural Language Processing Datasets, Audio Speech and Music Datasets for Machine Learning Projects, Data Visualization Datasets. You can check out other free source of datasets for machine learning with V7’s 65+ Best Free Datasets for Machine Learning.

However, the downside is that those open sources are not credible enough, so if your team accidentally gathers wrong data, your ML project will be affected badly, which reduces the level of accuracy for end-users. Also, collecting the data from unknown sources will cost you a great deal of time as it requires a lot of physical and manual labor.

So, the optimal strategy to get high-quality data for the task of labelling is to outsource to a professional vendor who has profound experience and knowledge providing data collection service to AI-based projects.

For your information, Lotus Quality Assurance is an expert at both data collection and annotation services. The datasets that Lotus Quality Assurance collects, including but not limited to images from reliable sources on the Internet, videos and sound captured and recorded with specific scenes, are provided with best quality and accuracy. 

If you have any difficulties in data collecting or data annotation for your projects, feel free to reach out to us!

Lotus Quality Assurance (LQA)

Tel: (+84) 24-6660-7474
Email: [email protected]
Website: https://www.lotus-qa.com/

Automated TestingManual TestingMobile AppSoftware TestingWeb App

Differences between mobile app testing and web testing

As we all know, technology is growing and mobile phones, smart devices are the trend, the future of the world. To meet such a large demand, applications must be of good quality, with rich and various content to meet the needs of users. That is also a big challenge for developers and software testers, especially for testers who are familiar with web testing but not yet aware of mobile testing.

So how is mobile testing different from web testing? 

1. Mobile testing has more flatforms

Mobile application testing will be much more complicated due to the variety of mobile devices. Ensuring that mobile applications work on all types of devices (smartphones, tablets or phablets) provided by a number of major vendors (such as Samsung, Sony, Nokia, HTC, Apple …) and on Operating systems (iOS, Android, Windows, Blackberry …) is really a challenge.

Therefore, testers need to have as many test cases as possible for each version, and perform as many tests on different devices as possible for good mobile application.

2. Screen size is constantly changing

iPhone-screen-sizes-mobile-web-testing

Major suppliers tend to change the size of their phones, tablets or phablets to fit users’ needs and tastes, as well as compete with other vendors. Then, when testing, we need to have test cases for screens of different sizes, to check for different screen sizes, whether the interface has broken layout or not; size buttons, the text box, radio button … have been changed or not.

3. User experience is needed for tester

For developers they only care about how the system functions work properly but for testers it is required user experience to support the users. An application can be considered a failure if it is difficult to use or can not be used in different situations, especially mobile applications need to support users to be able to use in all circumstances.

4. Users have more interactions than web applications

interaction-mobile-testing

When testing the web, the testers only interact with the system through a keyboard or mouse. But for mobile apps, we will interact with more ways such as touching, waving, eye movement, sound …..That’s why when testing, we will need to grasp catch those test cases.

5. Data security and privacy

Mobile applications such as pictures, videos are needed to be inaccessible to many other flatform feature. For mobile testing, there are also test cases of data access and privacy like camera access, photo data access, etc.

6. Excessive reliance on emulators and simulators leads to a lack of real-life device experiences

emulator-mobile-testing

When we test websites, the differences between browsers are quite small. However, for mobile apps, we often test in emulators and simulator environments. Those environments are not like real devices. Therefore, there are test cases that cannot be performed in emulators and simulator environments. As a tester, we need to control such cases.

7. Cases about installation, removal, update versions of application

Mobile applications are installed, removed and updated more frequently so that when testing, we need to catch how the next platform version changes and what it can affect the application.

remove-app-mobile-testing

What if the users have more than 1 device? What if those devices have different versions of the application? Compatibility, simultaneous support for multiple versions, data storage and the ability to install/ upgrade multiple times,… play an important part in application testing. 

8. Does the app work well when being interrupted

For mobile applications, users will often be interrupted by other applications such as SMS, incoming calls, notifications. After finishing those interruptions, how will the app work save actions in progress? These cases are often very rare when we test the web.

incoming-call-mobile-testing

9. Phone’s specific functions testing

There are many factors that need to be taken into account when testing.

・How much data is this application consuming?

・How much battery does this application consume? 

・Does this application work well if the battery is low? 

・How much junk data does it generate? 

—————————————————————

For more information about testing services, please refer to Testing services

Lotus Quality Assurance (LQA)

Tel: (+84) 24-6660-7474

Email: [email protected]

Website: https://www.lotus-qa.com/

LQA NewsLQA NewsLQA NewsLQA NewsLQA News

CEO Xuan Phung’s Interview (P2): Some surprising facts about her personal life

Continuing with the interview, this time, I want to ask Xuan about her personal life because I believe a a busy leader like Xuan are very good at work-life balance.

 

Thank you for sharing about the work you’ve done. May I ask about your personal life? What is your childhood dream?

When I was a little girl, I wanted to become a painter because my father was a painter. And he made a lot of wonderful pictures. When it came to decide which university to apply for, my father asked if I wanted to become an artist or an engineer. I chose to become an engineer and I did not regret the decision. I love to create technology and want to work full time for it while painting can be my hobby, which means I can paint in my free time.

 

What do you think your characters are?

Ummmm…! What a hard question! I think I’m the kind of person who never gives up, very ambitious. People around me also say the same.

 

 

Who do you respect the most and why?

This is another hard question. I respect everyone because they are different from me and I can learn a lot from these differences. But the person I respect the most is my mother. My mother is the leading example of a person who “never gives up”. My mother brought me up in a challenging situation but she succeeded in raising a great family.

 

What do you do on weekends?

I do two types of activities. To relax, I go out with my friends for a coffee or spend time with my children, or read books, go shopping, so on! I do many things that would make me happy and gain more energy for the new week. Apart from entertainment, I also do some work-related stuff such as reviewing and making plan for the new week. 

 

 

What do you like about Japan?

I like everything about Japan. The people, the food and the surroundings. The first time I’ve been to Japan, I got lost and an old man took me from the airport to my dorm. It was a long distance but he still offered to help me. I felt very lucky at that time. Japanese foods are also very delicious, and the environment is also very clean and clear.

 

Thank you for sharing your thought. Last but not least, I would like to ask about your typical day. As the CEO of LQA, I think you have a very busy personal life, I am curious how you can manage.

 

It’s boooring (smiley face). 

4:30: I get up then quickly check my work such as email, schedule of the day.

5:30 – 6:30: Go to the gym or studying Japanese.

6:30 – 7:30: I make breakfast and enjoy it with my family.

7:30 – 8:00: I take my daughter to school and go to office. 

8:00 – 18:30: Work. 

18:30 – 21:30: I make dinner, play with my children if I don’t need to go out for dinner with clients.

21:30 – 23:00: I work a little bit if have a conference calls or reading book.

23:00: I do some personal stuffs then go to sleep.

You’re such an early bird. What a hectic life! That’s not boooring at all (smile). I really admire you. Thank you very much for finding time in your busy schedule to have a talk with me. I wish you best of luck.

 

From this interview, I learn more about LQA and why LQA was able to grow that fast. I also understand more about Xuan a very passionate and thoughtful person. With Xuan san leading the company, LQA is on the road to success.

If you have not read the Part 1 of this interview, please check it out here

 

Watch our introduction video: LQA Introduction


Lotus Quality Assurance (LQA)

Tel: (+84) 24-6660-7474
Email: [email protected]
Website: https://www.lotus-qa.com/

 

BlogBlogBlogBlogBlogBlogBlogMobile App

Mobile application testing-Mobile testing tutorial 2

Mobile application testing is 1 type of mobile testing. For more information, refer Mobile testing in series of Mobile testing tutorial

1. Mobile application categories

For mobile applications, it can be classified into three categories:

・Type 1: Native apps – apps written specifically for a platform like iOS, Android, or Windows Phone in their respective languages.

・Type 2: Web application (Web App) – web-based application, mobile device users will use different browsers such as Chrome, Firefox, Safari to access the web server to use, such as m.facebook.com.

・Type 3: Hybrid App – a combination of native application and web application, can be run both offline and online and often uses web-making techniques such as HTML5, CSS.

mobile-venn-mobile-testing

In view of this

・ Native applications can only run on a certain operating system while mobile web applications can run on all mobile browsers that support HTML and Javascript.

・The native application is written in platforms like the SDK while the mobile web application is written with web technologies like HTML, CSS, ASP.NET, JAVA, and PHP.

・ For a native application, it is necessary to install but with mobile web applications, there is no need to install.

・ The original application can be updated from the app store while the mobile web application is updated centrally.

・ The original application may work without an Internet connection, but for mobile web applications, an Internet connection is always required.

・ Native apps work faster when compared to mobile web apps.

2. Some special test cases for testing mobile applications:

・Battery consumption: It is important to track the battery drain when running applications on mobile devices.

・Application running speed: Track time response time on different devices with different memory capacities, different network speeds …

・Memory requirement: When downloading and installing an app, running the app ..

・Application’s function checking: To ensure the application does not crash when losing network connectivity or other external impacts.

3. Some automated testing tools for mobile applications

3.1. IOS has popular automatic testing tools like:

app-testing-tool-fonemonkey

FoneMonkey application testing tool

app-testing-tool-frank

Frank application testing tool

app-testing-tool-qtp

QTP application testing tool

3.2. Android operating system has some automated testing tools such as:

Robotium application testing tool

app-testing-tool-eggplant

Eggplant application testing tool (No free source)

For more information about testing services, please refer to Testing services


Lotus Quality Assurance (LQA)

Tel: (+84) 24-6660-7474
Email: [email protected]
Website: https://www.lotus-qa.com/

Automated TestingEmbedded TestingManual TestingSoftware TestingSoftware Testing

Mobile testing tutorial 1: Brief introduction

Mobile technology and smart devices are current trends and will change the future of the world. Millions of applications are downloaded daily from the Appstore or Google Play for personal devices. Mobile applications are various to meet the needs of education, health care or entertainment of users. In order to have quality products that meet the needs of real society, mobile testing is a matter of great concern. It’s a vital factor in the development process that brings to market the high-quality product.

generation-divided-on-mobile-mobile-testing

1. What is meant by mobile testing?

Mobile testing is a process by which mobile devices is tested for its functionality, usability and consistency,…..to ensure quality products

See more Testing services

2. What are types of mobile testing

There are two types of mobile testing: hardware testing and software testing or mobile application testing

2.1. Hardware testing: includes testing of the processor, screen size, resolution, memory, camera, radio, Bluetooth, WIFI, etc.

2.2. Software or Mobile Application Testing: also known as mobile application testing to distinguish it from the above method.

mobile-app-testing

3Differences in mobile testing compared to other types of testing:

3.1. The variety of mobile devices: 

Different device manufacturers such as HTC, SamSung, Apple, Nokia … with different screen sizes and hardware configurations

・Multi-platform (iOS 6,7,8, Android 4.2; 4.3; 4.4, BB 5; BB6 …)

・Mobile devices have different application runtime

3.2. Challenging hardware of the device

・Processing speed limit

・Device memory capacity limit

・WAP / HTTP device protocol differences

3.3. Challenges of network connection

・Different types of networks (GSM / GPRS / WIFI / 3G …)

・No predictable time for data transmission

・Difference in connection speed via physical

・Diverse network operators with different network features

3.4. Types of testing

In addition, mobile application testing also includes the following types of testing:

  1. UI Testing: Test UI, Menu style, consistency of user interface on different devices
  2. Function Testing: Functional testing of major mobile application functions according to the device’s specification
  3. Performance and Load Test: Test mobile app behavior in low resources (Memory/ Storage Space) and when multiple users access the mobile app site together
  4. Usability Testing: Checking the usability aspects of mobile applications
  5. Compatibility Testing: Testing the compatibility of your application with native device features to ensure that your application does not interfere with other applications in the device.
  6. Interrupt checking: Because mobile devices have much lower memory than desktops, make sure that when there are voice calls, SMS messages, charging plugs, low memory notifications while the app is running does not cause any conflict.

mobile-testing-solution

For more information about testing services, please refer to Testing services


Lotus Quality Assurance (LQA)

Tel: (+84) 24-6660-7474
Email: [email protected]
Website: https://www.lotus-qa.com/

Blog

CEO Xuan Phung’s Interview (P1): A deeper insight into her work and LQA

The New Year 2020 comes with a lot of plans and opportunities for LQA. Today, let’s have an interview with Mrs Xuan Phung – the ambitious captain of the ship LQA, to know more about LQA’s goals for this year as well as her life as a businesswoman. 

Happy New Year Mrs Xuan. Thank you for taking the time to talk to me. Firstly, can you share with me about your ultimate goal when you started the company?

We aim to make LQA a global Quality Assurance (QA) company. We want to provide talented QA engineers to companies worldwide. We also want to support clients in making a high quality software products so that the end-users can use it without any errors. Last but not least, I want to make LQA a second home for my teammates where they can continuously grow up professionally and have a happy life.

What were your difficulties when you started your company?

The first difficulty I met was finding the first client. In the beginning, there was only me, no staff.  I faced a lot of obstacles, but I became more familiar with tough situations. I think solving problems and dealing with difficulties are my actual job. At the moment, I don’t think of those challenges as difficulties but as part of my responsibilities to grow the company.

Could you share with me the strength and weakness of this company?

Our strengths are that we specialize in testing, and we spend time and effort in making our services better. Since LQA is becoming more global, the company can cooperate with testing companies in different countries to understand the specific quality standard in these countries. Our team are talented and fast learners with strong determination. Our engineers have international testing certifications (ISTQB). Furthermore, our staff can communicate in many languages such as English, Japanese and Korean. LQA’s weakness is that LQA is a young company, so there are many things to learn as we go. But we always try to learn from each other.

What do you think is the best asset of this company?

I think human capital, my teammates are LQA’s best asset because great staff make quality services and satisfy the clients.

Why do you choose Japan to expand the business this year?

Made in Japan becomes a famous brand representing high quality. We believe that  as LQA offer high quality products, we can also provide our services to Japanese customers. Moreover, I think Japan is a potential market for ITO (Information Technology Outsourcing) because the country lack of human resources, especially IT engineers.

The rest of the interview about Xuan will be published soon. Stay tuned!

Watch our introduction video: LQA Introduction


Lotus Quality Assurance (LQA)

Tel: (+84) 24-6660-7474
Email: [email protected]
Website: https://www.lotus-qa.com/

Data AnnotationData Annotation

LQA Client’s Testimonial: “LQA has been one of our best experiences when working with external annotation teams”

“We enjoy working with LQA because of the high quality of their work and their flexibility in accommodating any new task. In the past year, we had a variety of different projects, from simple bounding box annotations to complex pixel-wise segmentation, and every time the team was able to perform the task according to the specification and within the agreed time frame. We are very impressed with the amount of effort the team put into understanding of precise requirements and making sure there are no grey areas in the task before starting the work. The work processes seem very smooth and well organised, making the interactions easy and predictable. So far LQA have been one of our best experiences when working with external annotation teams.” – Daedalean

“Daedalean (www.daedalean.ai) was founded in 2016 with an aim to specify, build, test and certify a fully autonomous sensor and autopilot system that can reliably and completely replace the human pilot. Currently the company is working with EASA on an Innovation Partnership Contract to develop concepts of design assurance for neural networks.”

If you have any difficulties in data collecting or data annotation for your projects, feel free to reach out to us!


Lotus Quality Assurance (LQA)

Tel: (+84) 24-6660-7474
Email: [email protected]
Website: https://www.lotus-qa.com/

6 Annotation Types: What is the difference?

Data Annotation is the process of labelling the training data sets, which can be images, videos or audios. Needless to say, AI Annotation is of paramount importance to Machine Learning (ML), as ML algorithms need (quality) annotated data to process. 

In our AI training projects, we use different types of annotation. Choosing what type(s) to use mainly depends on what kind of data and annotation tools you are working on.

Bounding Box: As you can guess, the target object will be framed by a rectangular box. The data labelled by using bounding boxes are used in various industries, and most used in automotive vehicle, security and e-Commerce industries. 

Polygon: When it comes to irregular shapes like human bodies, logos or street signs, to have more precise outcome, Polygons should be your choice. The boundaries drawn around the objects can give an exact idea about the shape and size, which can help the machine make better predictions.

Polyline: Polylines usually serve as a solution to reduce the weakness of bounding boxes, which usually contain unnecessary space. It is mainly used to annotate lanes on road images.

3D Cuboids: The 3D Cuboids are utilized to measure the volume of objects which can be vehicles, buildings or furniture. 

Segmentation: Segmentation is similar to polygons but more complicated. While polygons just choose some objects of interest, with segmentation, layers of alike objects are labeled until every pixel of the picture is done, which leads to better results of detection.

Landmark: Landmark annotation comes in handy for facial and emotional recognition, human pose estimation and body detection. The applications using data labeled by landmark can indicate the density of the target object within a specific scene. 

If you have any difficulties in data collecting or data annotation for your projects, feel free to reach out to us!

The Governor of Kanagawa Prefecture: ”We are looking forward to LQA’s existence in Kanagawa”

On 19th November, LQA was honored to join “Kanagawa business seminar” – the significant networking event with the participation of Kanagawa government officials, Kanagawa and Vietnamese businesses.

 

Here, LQA CEO Xuan Phung had a conversation with Mr. Yuji Kuroiwa (黒岩 祐治), the Governor of Kanagawa Prefecture, to share with him about LQA’s plan to set up a Japanese subsidiary in Kanagawa. Kanagawa is an ideal place for the establishment, as it offers a range of advantages. In geographical terms, Kanagawa’s capital city – Yokohama, which lies on Tokyo Bay, in the Kantō region of the main island of Honshu, is a major commercial hub of the Greater Tokyo Area. The prefecture’s government also gives special preference to foreign businesses. 

 

 

Mr. Yuji Kuroiwa was delighted with this plan of LQA, and he believed that this would be a great step forward for the company. The government will be willing to help LQA so that the company can develop, achieve much success in Japan. 

 

 

 

LQA News

LQA OLYMPIC 2019: Work hard, play harder

Last Friday, LQA’s sporting event “LQA Olympic 2019” took place successfully, attracting the participation of all LQA-ers and some closed clients. In our 3rd anniversary, to encourage our employees to practice sports to have a healthier life and improve the solidarity of the teams, we organized a lot of team-building and sporting activities that bring joy and relaxation to our members. 

One of LQA’s missions is to build long-term career paths and providing a balanced work-life for our employees. That is the reason why internal bonding activities are a crucial part of our HR plan. This year, in addition to the Summer vacation trip, Mid-autumn festival celebration, …, our BOD decided to hold the biggest sporting event “LQA Olympic”. All of these activities have a positive impact on our staff’s mental and physical life and their relationship.

Taking a break from 9-5 grind, Lotusians came to a nearby football ground to join the event. Despite the scorching hot weather, all of us were excited to be part of the team and played in extremely high spirits. This was a chance for us to bond up the relationship with our teammates. And we could also connect with colleagues who we had never had a chance to talk to before.

This is truly one of the most unforgettable memories of our time working in LQA. After LQA Olympic activity, hope that LQA-ers can relax and have more energy in work to help provide clients the best service.

* You can watch our video here.

In this 3rd anniversary, we wish that LQA has more years of success, luck and joy, and LQA will go global. Happy birthday to LQA!