💖💖作者:计算机毕业设计小途
💙💙个人简介:曾长期从事计算机专业培训教学,本人也热爱上课教学,语言擅长Java、微信小程序、Python、Golang、安卓Android等,开发项目包括大数据、深度学习、网站、小程序、安卓、算法。平常会做一些项目定制化开发、代码讲解、答辩教学、文档编写、也懂一些降重方面的技巧。平常喜欢分享一些自己开发中遇到的问题的解决办法,也喜欢交流技术,大家有技术代码这一块的问题可以问我!
💛💛想说的话:感谢大家的关注与支持!
💜💜
网站实战项目
安卓/小程序实战项目
大数据实战项目
深度学习实战项目
目录
- 豆角价格分析与可视化系统介绍
- 豆角价格分析与可视化系统演示视频
- 豆角价格分析与可视化系统演示图片
- 豆角价格分析与可视化系统代码展示
- 豆角价格分析与可视化系统文档展示
豆角价格分析与可视化系统介绍
《基于大数据的豆角价格分析与可视化》这个系统主要围绕豆角价格数据展开,把采集到的价格记录放到HDFS里,再用Hadoop和Spark做处理,中间会用到Spark SQL、Pandas、NumPy做清洗、汇总和统计分析,后端用Python的Django来实现,也支持Java的Spring Boot版本,前端用Vue、ElementUI、Echarts、HTML、CSS、JavaScript和jQuery把结果展示出来,数据库用MySQL保存基础数据和分析结果。系统功能包括系统首页、大屏可视化、用户管理、豆角价格、数据分析、价格趋势分析、价格对比分析、价差关系分析、季节规律分析、波动聚类分析、渠道画像分析、个人信息和修改密码。用户可以从时间、市场、渠道、季节等角度查看豆角价格变化,看到趋势线、对比柱状图、价差关系、季节规律、聚类结果和渠道画像,也能在后台维护价格数据。整个系统重点是把大数据处理流程和可视化展示串起来,让原本零散的豆角价格记录变得能查询、能比较、能分析,适合作为计算机专业大数据方向的毕业设计。
豆角价格分析与可视化系统演示视频
项目演示视频
豆角价格分析与可视化系统演示图片
豆角价格分析与可视化系统代码展示
spark=SparkSession.builder.appName("DouJiaoPriceAnalysis").master("local[*]").config("spark.sql.shuffle.partitions","4").getOrCreate()defprice_trend_analysis(start_date,end_date,market_name=None):base_sql="SELECT stat_date, AVG(price) AS avg_price, MIN(price) AS min_price, MAX(price) AS max_price, COUNT(*) AS sample_count FROM doujiao_price WHERE stat_date BETWEEN '{}' AND '{}'".format(start_date,end_date)ifmarket_name:base_sql=base_sql+" AND market_name = '{}'".format(market_name)base_sql=base_sql+" GROUP BY stat_date ORDER BY stat_date"trend_df=spark.sql(base_sql)trend_df.createOrReplaceTempView("tmp_price_trend")result_df=spark.sql("SELECT stat_date, avg_price, min_price, max_price, sample_count, LAG(avg_price, 1) OVER (ORDER BY stat_date) AS last_avg_price, ROUND((avg_price - LAG(avg_price, 1) OVER (ORDER BY stat_date)) / LAG(avg_price, 1) OVER (ORDER BY stat_date) * 100, 2) AS change_rate FROM tmp_price_trend ORDER BY stat_date")result_df=result_df.na.fill({"change_rate":0})result_df.createOrReplaceTempView("tmp_trend_result")final_df=spark.sql("SELECT stat_date, avg_price, min_price, max_price, sample_count, change_rate, CASE WHEN change_rate > 5 THEN '上涨明显' WHEN change_rate < -5 THEN '下降明显' ELSE '相对平稳' END AS trend_label FROM tmp_trend_result ORDER BY stat_date")mysql_url="jdbc:mysql://localhost:3306/doujiao?useSSL=false&characterEncoding=utf8"final_df.write.mode("overwrite").format("jdbc")\.option("url",mysql_url)\.option("dbtable","price_trend_result")\.option("user","root")\.option("password","123456")\.save()returnfinal_df.collect()defprice_compare_analysis(compare_type,start_date,end_date):ifcompare_type=="market":compare_field="market_name"elifcompare_type=="channel":compare_field="channel_name"else:compare_field="variety"compare_sql="SELECT {} AS compare_name, AVG(price) AS avg_price, MAX(price) AS max_price, MIN(price) AS min_price, STDDEV(price) AS std_price, COUNT(*) AS sample_count FROM doujiao_price WHERE stat_date BETWEEN '{}' AND '{}' GROUP BY {}".format(compare_field,start_date,end_date,compare_field)compare_df=spark.sql(compare_sql)compare_df.createOrReplaceTempView("tmp_price_compare")rank_df=spark.sql("SELECT compare_name, avg_price, max_price, min_price, std_price, sample_count, RANK() OVER (ORDER BY avg_price DESC) AS price_rank, ROUND(avg_price / (SELECT AVG(avg_price) FROM tmp_price_compare) * 100, 2) AS relative_index FROM tmp_price_compare")rank_df.createOrReplaceTempView("tmp_price_rank")final_df=spark.sql("SELECT compare_name, avg_price, max_price, min_price, std_price, sample_count, price_rank, relative_index, CASE WHEN relative_index >= 110 THEN '偏高' WHEN relative_index <= 90 THEN '偏低' ELSE '正常' END AS price_level FROM tmp_price_rank ORDER BY price_rank")mysql_url="jdbc:mysql://localhost:3306/doujiao?useSSL=false&characterEncoding=utf8"final_df.write.mode("overwrite").format("jdbc")\.option("url",mysql_url)\.option("dbtable","price_compare_result")\.option("user","root")\.option("password","123456")\.save()returnfinal_df.toPandas().to_dict("records")defseason_rule_analysis(start_year,end_year,variety_name=None):where_sql="WHERE year(stat_date) BETWEEN {} AND {}".format(start_year,end_year)ifvariety_name:where_sql=where_sql+" AND variety = '{}'".format(variety_name)season_sql="SELECT year(stat_date) AS year_num, month(stat_date) AS month_num, CASE WHEN month(stat_date) IN (3,4,5) THEN '春季' WHEN month(stat_date) IN (6,7,8) THEN '夏季' WHEN month(stat_date) IN (9,10,11) THEN '秋季' ELSE '冬季' END AS season_name, AVG(price) AS avg_price, STDDEV(price) AS std_price, MAX(price) AS max_price, MIN(price) AS min_price, COUNT(*) AS sample_count FROM doujiao_price {} GROUP BY year(stat_date), month(stat_date), CASE WHEN month(stat_date) IN (3,4,5) THEN '春季' WHEN month(stat_date) IN (6,7,8) THEN '夏季' WHEN month(stat_date) IN (9,10,11) THEN '秋季' ELSE '冬季' END".format(where_sql)season_df=spark.sql(season_sql)season_df.createOrReplaceTempView("tmp_season_detail")summary_df=spark.sql("SELECT season_name, AVG(avg_price) AS season_avg_price, AVG(std_price) AS season_std_price, SUM(sample_count) AS total_sample, MAX(max_price) AS season_max_price, MIN(min_price) AS season_min_price FROM tmp_season_detail GROUP BY season_name")summary_df.createOrReplaceTempView("tmp_season_summary")final_df=spark.sql("SELECT season_name, season_avg_price, season_std_price, total_sample, season_max_price, season_min_price, RANK() OVER (ORDER BY season_avg_price DESC) AS price_rank, ROUND(season_std_price / season_avg_price * 100, 2) AS volatility_rate FROM tmp_season_summary ORDER BY price_rank")mysql_url="jdbc:mysql://localhost:3306/doujiao?useSSL=false&characterEncoding=utf8"final_df.write.mode("overwrite").format("jdbc")\.option("url",mysql_url)\.option("dbtable","season_rule_result")\.option("user","root")\.option("password","123456")\.save()returnfinal_df.collect()豆角价格分析与可视化系统文档展示
💖💖作者:计算机毕业设计小途
💙💙个人简介:曾长期从事计算机专业培训教学,本人也热爱上课教学,语言擅长Java、微信小程序、Python、Golang、安卓Android等,开发项目包括大数据、深度学习、网站、小程序、安卓、算法。平常会做一些项目定制化开发、代码讲解、答辩教学、文档编写、也懂一些降重方面的技巧。平常喜欢分享一些自己开发中遇到的问题的解决办法,也喜欢交流技术,大家有技术代码这一块的问题可以问我!
💛💛想说的话:感谢大家的关注与支持!
💜💜
网站实战项目
安卓/小程序实战项目
大数据实战项目
深度学习实战项目